Are we really related to Robert E. Lee?

The family story on my mom's side says we are related to Robert E. Lee. No one has proved it yet, and I have read that most of the time people are not related to him.

If we were related then it would probably be through Juda Frances Lee who married William Thomas Hardee.

Juda's parents were Burrell and Lucinda Ramsey Lee. They married December 31, 1833 in Clark Co., Georgia. They had six daughters; Ruthy born about 1835 in Georgia, Lucinda born about 1837, Juda born about 1837/8 (twins maybe), Sarah born about 1841, Jane born about 1842, Mary born about 1844. All but Ruthy were born in Alabama.

Lucinda is found in the 1850 census at Hatchett Creek District, Coosa Co, Alabama (1)
Lucinda        Lee    36 F    Spinster        born in Georgia
Ruthy           Lee    15 F                        born in Georgia
Lucinda       Lee     12 F                        born in Alabama
Judy            Lee     12 F                        born in Alabama  (My line)
Jane            Lee       8 F                        born in Alabama
Mary           Lee      6 F                        born in Alabama

A daughter named Sarah is not listed in this Census. I do not know where she is. Need to look for her. I have found her in the 1860 census with her mother Lucinda and Mary living with her and her husband and baby boy.

Burrell is a mystery. I do not know what happened to him. Why is Lucinda listed as a spinster? A few definitions for spinster are an unmarried, divorced, or widowed women, or a person who spins fibers. She wasn't unmarried but could have been divorced or widowed. She could have been a spinner too, who knows.

I found a Lee in the 1840 Census, Talladega Co., Alabama listed as Bunell:

Name: Bunell Lee [Burrell Lee]
Home in 1840 (City, County, State):     Talladega, Alabama
Free White Persons - Males - 20 thru 29:     1  [Burrell]
Free White Persons - Females - Under 5:     3  [Lucinda, Juda, Sarah]
Free White Persons - Females - 5 thru 9:     1  [Rutha?]
Free White Persons - Females - 20 thru 29:     1  [Lucinda]
Free White Persons - Under 20:                4
Free White Persons - 20 thru 49:               2



I know Burrell and Lucinda were in Alabama by 1838 when Lucinda was born so they would most likely be in the Alabama 1840 census.  I am unsure if this is my Burrell Lee but I have added names in brackets that correspond to the family members. I added Sarah to the Females under 5 but I am not sure if she was born by then, so this may be incorrect. Just another clue till I find more information.

Some Ancestry trees show Burrell's father as Wyatt Lee of Clark Co. Georgia. I found Wyatt's obituary and it states he had 26 children. I have looked into his children and contacted other researchers who have information on Wyatt and none show a son named Burrell, so I do not believe Wyatt was his father.

So until I can find out more about Burrell Lee and who his parents were we will not know if we are related to Robert E. Lee or not.

(1) 1850 U.S. census, Coosa Co., Alabama, population schedule, Hatchet Creek District, p. 114b, dwelling 365, family 365, Lucinda Lee; digital images, Ancestry (www.ancestry.com); citing National Archives and Records Administration microfilm M432.