The Son Also Rises: Surnames and the History of Social Mobility

Your name can tell a lot about you and your economic prospects. Theoretically, in an egalitarian society, all families would revert to the mean. Those that were rich would have offspring that were somewhat advantaged, but over the generations they would be similar to others. Similarly, the poor would have the opportunity through education to increase their stock. However, in looking at surnames and associated markers of prosperity, there is an unexpected stickiness of prosperity.

Many of the tests done involved correlating names with indicators of "success". This was somewhat limited by the available sources. In Britain, the authors looked at Members of Parliament, students at Cambridge and Oxford, and probate records. The focus was primarily on the more rare surnames. It was found that surnames that were overrepresented by wealth markers tended to stay over-represented. Their reversion to mean was slower than would be statistically expected. This was found even when large investments were made in education. Similar results held in almost all other countries, even those that were supposedly very egalitarian. Sweden provides free education and subsidizes living expenses, yet the rich continue to hold their positions. Even after the cultural revolution in China, the elites came back. Similarly the descendents of samurai in Japan continued to show privilege long after the positions were abolished. These factors seemed to show something even beyond genetics or nurturing.

The authors did acknowledge there were some challenges to the methodology. Some people would change their names. (Though this relatively small occurrence would often reinforce the elite positions.) It can also be difficult to get accurate data, with many places recording things differently (and the same places have different data over time.) 

There were also some interesting results that appeared for certain groups. Those with European French surnames in the US tend to be underrepresented as doctors. (Perhaps because it was the poor that most often immigrated.) Jews are over-represented among the elites. This may have started as they had widespread literacy before others, yet it continued after the world caught up. In Ireland, there was a sorting out of Catholics and Protestants with people often sorting to their class group.

Analyzing groups as individuals is often subject to controversy in our world. It does provide some interesting insight into different values and changes among the people of the world. 

Family Search and Python

Family Search has a huge interlinked collection family tree and genealogy collection. I thought it would be fun to answer a few questions:

• Where are my ancestors from?
• How have family sizes changed over generations?
• Have there been lines that have grown large, but left with no descendents.
• How far back do you need to go before finding the same ancestor in different lines.
• How many years is a generaation.

First step was to see what was there. FamilySearch solutions gallery contains a number of different third party tools. Some look a little sketchy, but others seem fairly well done.

Map My Cousins shows an animated map of ancestors and migration.

There was another tool that would show a pie of your origins. The problem is that there are a lot of "gaps". (A big chunk of genealogy was "unknown".) It does have the option to "guess". However this just seems to apply the same place to the parents as the children, thus leading to an overrepresentation of USA.

Time to look at the API. Alas, FamilySearch has an API, but it is not easily accessible. They seem to focus on companies rather than allow individual use. There is some mention of development program that might work. Though it seems they will only give you access to sample data at first. You then must have a company and pay $200 per app before getting real access.

So if I can't use the API, at least I could download my genealogy and then analyze it, right?  Unfortunately, there is no download option. They suggest using a third party tool, but don't recommend a specific one. I ended up using the free version of Ancestral Quest. It looks like a DOS program migrated to MAC. 

I could download up to 100 generations of data. However, it occasionally crashes. It seems to be certain lines that are more likely to cause crashes. To get around it, I loaded 100 generations back from each grandparent. Then I narrowed the problem line. Once I got errors about a circular ancestry. Maybe that was what caused the crash? It was odd that it worked one time.

It has the ability to export data as Gedcom. I found a JavaScript parser, but it was broken. I ended up going for a python Gedcom parser. Now the fun time learning Python!

My first exercise was to find lines that went back further than 100 generations. There was a method to get ancestors. However, it didn't seem to work. I had to get the hang of other methods performed on a parser object, vs. on individual elements. It took a little getting used to. I eventually did a basic tree iteration to find the "endpoints" that could go back further. 

The final results had some lineage tracing back about 200 generations. Things got sketchy as it went further back. I noticed one line went from Vikings to biblical Jewish ancestry. Some long lines went through to ancient Egypt and Mesopotamia. 

For the next step, I wanted to try to go forward from a few generations back. Getting the data loaded was a much more painful process. I tried to get 11 generations and go back from there. I first started getting everybody. That was overkill. AncestralQuest does dedup, but it explores everything first. I realized I could shortcut it by just doing the husbands. Due in part to maternal mortality, it would be more likely for a man to have multiple spouses. Other than that, they would both have the same descendents. It still was painfully long. I did notice that some lines were very complete. These would take hours to download the descendancy from a single 11-generation old ancestor. (And return almost 100k people.) Other seemed to be a tentacle that reached back. One line crashed while downloading. Uggh.

Eleven generations is a painful amount to get. You don't have access to other living people, the more recent generations are sparse. Going further back and the records are missing, making those sparse.

Now on to coding. First I parsed the file and then traversed the tree starting at the root to find the furthest back end-nodes. Then from each of those I gathered the families and then children for each generation. I needed to keep a set of ones I had seen to not get into bad loops. It seems there are some duplicates of the same person in different places. This makes it hard to distinguish from "real" common ancestors.

The average years per generation seemed to be right around 30 years.

Same Ancestor in different lines

This seemed fairly straightforward. Just store the path to each ancestor that we find. If we find the same person twice, compare the path to see how we got there.

Things got funky when I started looking at some lines that went way back. Two lines had a common ancestor. However, one had about 16 generations in between, while the other had more than 50. The birth years brought out the funkiness. On the one with more generations, it went back to the 1400s, then a few without birth years followed by some in more modern times. Looks like bad data.

Some of the other cases were possibly also bad data. It looked like the same name with a few different birth years.

There were a few "real" common ancestors. Going back a few centuries, there were some small-town residents that had common ancestor only a few generations back. There were also a few cases where different lines converged on common ancestors about 15 generations back.

Time Dimensional Maps

Maps are very good at showing a given place at a given time. Modern interactive maps allow easy exploration of two or three dimensions of the "snapshot". However, they don't do a very good job of showing time.

I would like to be able to visualize my ancestors and how they interacted and participated in historical events. A few issues come up with this:
1) How do you find the places?
Modern map companies do a great job of geocoding almost any address to its appropriate latitude and longitude. Past addresses are much more of a challenge. Using a database of modern names can probably get a lot of places (Boston is still Boston.) However, other place names have changed (where is Prussia?) Or even more confusing, some names may refer to something different today.
Some of this is somewhat mitigated by lack of detailed data as you travel back in history. (You may just have a county or even just a country.)
Some other data exists with Township/Section/Range descriptions. This could theoretically be converted to map locations, but takes a little work.
2) How do you represent time on a map?
You could color code different points to represent different time periods (years, decades, centuries or whatever is appropriate.) This allows for visualizing 'neighbors' in time, and associating them with events. (The person living in Boston in the 17th century probably didn't participate in the American Revolution...)
Another possibility is tracks. You could display the movement of individual people across time. With ancestry, there is also a clear linkage. This could provide a picture of migration.
With all of these approaches, the data representation could change as zoom level is changed, thus giving it an uncluttered approach.

With any approach, an important part is ease of use. Importing gedcom files would be a bare minimum approach. It could also be interesting to parse census data to show specific household migration (though this would be much more involved.) Getting township/section/range data on the map should also be automatic. A public location repository would also be useful. (Once you figure out where a location is in 1650, others should be able to use it.)