The children's mother was home at the time of the shooting Tuesday afternoon but had stepped on to the porch for "no more than three minutes," Cumberland County Coroner Gary White told WKYT-TV.
White told the Lexington Herald-Leader the boy received the .22-caliber rifle as a gift. He said the rifle was kept in a corner and the family didn't realize a bullet was left inside it.
"It's a Crickett," White said, referring to a company that specifically makes guns, clothes and books for children. "It's a little rifle for a kid. ... The little boy's used to shooting the little gun."
White said the shooting was an accident.
It wasn't immediately clear who gave the boy the gun or exactly what led up to the shooting. White did not return a telephone call from The Associated Press on Wednesday.
State police said in a brief news release the shooting occurred when the boy was "playing" with the rifle, but did not elaborate.
It is not clear whether any charges will be filed, said Kentucky State Police spokesman Trooper Billy Gregory.
"I think it's too early to say whether there will or won't be," Gregory said Wednesday.
Keystone Sporting Arms, based in Milton, Penn., produced 60,000 Crickett and Chipmunk rifles in 2008, according to its website. It also makes guns for adults, but most of its products are geared toward children.
The company's slogan is "my first rifle" and its website has a "Kids Corner" section where pictures of young boys and girls are displayed, most of them showing the children at shooting ranges and on bird and deer hunts. The smaller rifles are sold with a mount to use at a shooting range.
"The goal of KSA is to instill gun safety in the minds of youth shooters and encourage them to gain the knowledge and respect that hunting and shooting activities require and deserve," the website said.
No one at the company answered the phone Wednesday.
According to its website, Bill McNeal and his son Steve McNeal decided to make guns for young shooters in the mid-1990s and opened Keystone in 1996 with just four employees, producing 4,000 rifles that year. It now employs about 70 people.
Burkesville sits near the Tennessee-Kentucky state line along the Cumberland River, among the foothills of the Appalachian Mountains. The small city is about 90 miles northeast of Nashville, Tenn.
It is home to a Mennonite community that gained attention in 2010 when nine of its members were killed in a head-on collision with a tractor-trailer.