Sunday, June 25, 2017

The Swarm: The Second Formic War (Volume 1)

I didn't realize this series of books could fit in the Endger's Game timeline. I had thought that the first Formic War was immediately followed by Ender's Game. However, there is the matter of Mazer Rackham having been traveling through space to be able to talk with Ender, so I guess it makes sense. This books feels like the previous Formic War series. There are some people that see what is going on. Those with power don't believe them and dilly dally until it starts to get too late. Most leaders are greedy and really in it for their own benefit. They feel threatened by people that are more capable precisely because they are more capable. The "heroes" are crazy smart, but suffer from self doubt and can be a little bit too humble. (And how is it, that the Aliens always happen to be near mining ships that know how to identify them?) Mazer is ignored and in trouble with the military. Young Jukes is working hard and tries to do the right thing, but not ruthless like his father. Victor can fix anything and is brave enough to go into space. His fiance enlists in the military and soon finds herself in command of their now commandeered mining ship. Everyone seems to understand the political reason for their actions - except for the politicians themselves.
The author's seemed to go all George Lucas on the story. In the afterward, they acknowledged that there was not a whole lot of detail on Mazer in Ender's Game. Thus, they got to make an entire backstory to fulfill a couple small, nearly throwaway lines. It must have been a lot of fun to write the story. In reading it, you know how the main arc will turn out, but there are still plenty of sub-plots to keep things interesting. There are obviously going to be a few more books in this series.

Tuesday, June 20, 2017

Algorithms in a Nutshell: A Practical Guide

A programmer wanted to prevent memory leaks, so he wrote some code to store a record of all memory allocated and freed. However, this code resulted in the programs sometimes taking forever to run. Only after some analysis, it was determined that the binary tree used for storing memory locations was unbalanced, due to the sequential nature of memory allocation in malloc. Balancing the tree reduced the horrible worst-case run times.
This story lead to the promise of a great, practical book on algorithms. Alas, after starting off well, the book soon went of the deep end. Rather than providing simple algorithms to answer real world questions, it dove into deep analysis of algorithms with multiple tables of timings. (Hint: do not use text to speech unless you want to hear endless pronunciations of large numbers.) It went to provide very detailed analysis of implementations of certain algorithms on various platforms. Alas, it only covered a limited number of specific algorithms in this detail. It also went on to cover in significant depth some algorithms with limited use cases. The detail was both too much for a general "nutshell" view of algorithms and not nearly enough for a detailed reference book.

The Rule of Three: Fight for Power

The second Rule of Three book has less direct action and more internal politics. People have now lived for some time without electricity. They need to balance their need for survival with the needs to be ethical humans. The community does not have the resources to help everyone. How can you help people in need when your resources are extremely limited? In some cases, simply communicating and given out things you are not using is the key.
What do you do when people that exhibit valuable behavior also have the tendency to stretch things and act in ways detrimental to the community? Do you keep them around? Do you give them greater freedom to act how they desire, even if it may be outside the realm you want for society?
Unfortunately some of the community begins to unravel from within. The book tends to radio key points from a long way out. As a reader, I had suspicion as to the cause of certain key events. It seems obvious why events were happening, but the characters seemed to have no clue. However, the conclusion totally took me by surprise.

Tuesday, June 13, 2017

The Signal and the Noise: Why So Many Predictions Fail--but Some Don't

We spend a hug amount of money and computational power on predicting weather, yet we still complain about inaccurate weather forecasts. In spite of this, weather forecasting is one of the "success stories". We have much more accurate forecasts than we did prior to the computational advances. However, beyond a week, weather forecasts fare no better than guesswork. Small changes in variables can significantly alter the long term prognosis. People can also cause kinks in the operations. Commercial weather forecasts have a tendency to overestimate precipitation. (They would rather have someone pleasantly surprised by a sunny day than have an activity ruined by rain.) There was also a case of flooding in North Dakota where an accurate river level prediction was made. However, only the average number was shared rather than the range. The river crested within the range, which happened to be just above the flooding level (and above the average level predicted.)
Baseball provides a rich source of data about many players. It also provides many opportunities for inaccurate predictions about players. Successful teams use a mixture of scouting and statistical analysis to find the best players for the money.
There are a number of biases in the data analysis and predictions we see. Bold predictions are most likely to get press coverage, but are least likely to be right. It is almost always possible to find a significant pattern in the "noise", but that doesn't do much good for predicting future signals. People also tend to be really bad at understanding what data means. Furthermore, news coverage tends to focus on the outliers, even though they tend to be the most inaccurate.
Coverage of global warming provides a cautionary example. There is scientific consensus on the negative impact of human activities on the earth's climate. However, consensus does not necessarily mean good science. Rather than being a balanced average of different opinions, a consensus tends to be dominated by the loudest or most forceful voice. For climate change, the initial view was simply that a greenhouse effect existed and that human activity contributed to increasing in gases. After this point, things got wonky. Discussion switched to global warming, with precise numbers given for warming predictions. When these tended to overstate the warming, the predictions were revised down and models were calibrated. However, the more extreme predictions were the ones that received more press coverage. This would distort the public's view of the situation and give greater credence to the opponents. The response to global warming involves politics, and politics is concerned with the short term impacts, not the long term results. Thus, the noise ends up being twisted towards short term purpose, while the signal is left in the scientific circles.
Predicting terrorism is a lot like predicting earthquakes. We know it is likely to happen, with the minor activities being more frequent than the high-body-count ones. However, we are not good at knowing the specifics. The September 11, 2001 attacks were "unknown unknowns". They were just not something we expected or thought to expect. This made prediction difficult. Terrorism does tend to follow a power-law distribution giving us an idea that a terror attack might be due, but no more than that. (Ironically, Israel seems to buck the power law trend. They permit small-scale attacks to happen, but focus efforts on limiting the more damaging large scale attacks.)
The real problem with predictions is people. The sensational tends to be more appealing than well-thought out. A single pronouncement is given more weight than one couched in uncertainty - even though the uncertain one is much more truthful. People also tend to value "loyalty", giving more credit to those that stick by their guns, even though a willingness to change predictions in face of data makes for more valuable predictions. What are we to do?

Friday, June 09, 2017

Rule of Three

Rule of Three starts with a student working on his paper at the last minute. Then all power shuts down and things go haywire. The problem appears to be more widespread than just power - all computers (or anything containing them) are shut down. An explanation is never given for why. However, the power outage is widespread. Luckily, the protagonist drives an old car, so he is still able to get home. (He is also able to drive his "crush" home to her farmhouse.) Once home, he meets up with his neighbor, a former government operative who is also a survivalist.
The boy's mom happens to be a police officer and initially works to keep things in order. They arrange for people to "buy" goods on credit from local stores (thereby forstalling the looting attempts.) However, things quickly go downhill as the outage goes on. Eventually, they focus on their neighborhood. They set up security checkpoints, build a wall and recruit the local farmer (who happens to be the father of the girl that the boy likes) to come in and help farm plots. They seem to have a viable community.
The boy eventually gets his ultralight airplane to run and explores other areas. They discover the police station has been destroyed by a rocket launcher, probably by a group that has a Cessna. They also open up a mutually beneficial alliance with another community. Later, the Cessna group destroys the other community and murders anyone they can find. From there to book works towards its violent climax, complete with airplane fights and boy falling in love with girl.
It feels very similar to other "collapse of modern culture" novels, though this is told from a high school perspective.

Monday, June 05, 2017

How Will You Measure Your Life?

In How Will You Measure Your Life, Clayton Christiensen applies his theories to personal endeavors. He and his coauthors stress that this is not a self-help or business book with prepackaged conclusions, but instead a set of theories that can be applied as appropriate to the situation. Even a great theory may not be perfectly suited for each situation. But even the imperfections can be useful in illuminating the situation.
In work, people are motivated by intrinsic factors. Compensation, titles and benefits do not necessarily motive, but the lack thereof can result in a negative experience. People often get confused, chasing after the material rewards and find themselves miserable. If a family is important, time needs to be spent with family. Sometimes career choices need to be made with the long term perspective in mind - even if they seem short term in nature. Spending time with children at the youngest age is the most important. Trying to tack it on later cannot compensate for what is missed at the end.
When deciding on a course of action, one question that should be answered is "What is needed for this to succeed?" If the success criteria are not present, it will be difficult to have a successful outcome. In relationships, people often get into the trap of doing what they think somebody else would like to have done. They may work hard for something, think that is what they would want if they were in the other person's shoes. However, that may make the other person unhappy because that is not what they really wanted. This leaves two unhappy people. It is important to really know somebody else and spend the effort in service.
The value of work is important in family life. There is danger in outsourcing too much. Dell computer is a cautionary tale of a company that gradually outsourced its consumer business to Asus. Each move up the value chain seemed like a good move. Pretty soon, however, Asus was ready to do it on its own and didn't need Dell. (Luckily, Dell had other businesses to fall back on.) In family life, we outsource the raising of children by signing them up for many activities. However, having them spend time with us doing work may be even more valuable. People need to learn how to work hard and solve challenging problems. Being in prefabricated, easy to solve situations does not help that. Bring some of the tedious chores back "in house" and requiring children to be involved may be more beneficial. Children learn things when they want to learn, not when we are ready to teach them. The best way to instill are values is to always live them.
It is also important for us to uncompromisingly live our values. Once we start to compromise on what we believe, it becomes more difficult to continue to live them at later times. It good to spend time understanding what our values our and what is needed to live them. Failures to achieve goals are good and can help us to live a better life.
This book grew out of discussions in business school classes on how to define success. As such, many of the examples have a business twist. The methodology, however, is fairly universally appropriate, even if there is no "business" in the life.

Tuesday, May 30, 2017

Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are

Everybody Lies weaves together two interweaving threads. The first is that what people type in private in a search engine tells a lot more about their true feelings than what they tell to pollsters. (People will often search for "non socially acceptable" things such as racist or sexual keywords, but would not admit that to pollsters.) The second is that analyzing big data can provide us answers that we could not find using smaller data sets. An example of a finding was on crimes caused by violent movies. Using hourly crime data and violent movie box office, it was found there was a decrease in violent crimes when a popular violent movie was playing. This may be due to the violence-inclined watching the movie instead of getting drunk and violent that evening.

Most of the work delves in to the big data analysis of the non-socially acceptable topics. People would not tell a pollster something, but they would be willing to type it into a search engine. The difference between the public postings on Facebook and the private searches on Google would be an interesting avenue of exploration. (But, alas, the book doesn't go there.) The author acknowledges that there are weaknesses in using extremely large data sets. It is very well possible to find an answer that is merely coincidence. You can almost always find the answer that you are looking for, but need to be careful to make sure it is really true. (However, I wonder if he also falls victim to this also. If something is unacceptable in an area, the people publicly supporting it may be lower, while searches may be higher. However, would the reverse also be true, with people in an area where it is acceptable to say they support it, while not actually searching for it?)

Anti-muslim behavior provides an interesting case study. After attacks in San Bernadino, anti Muslim sentiment was on the rise. Obama tried to quell this by giving a speech stressing peace. The speech went over well with the media. However, a spike in Muslim-hate searchers occurred during the speech. The one time when it went down was when he talked about Muslim athletes. People were then more interested in searching out how Muslims were similar to them. This helped provide a base for future attempts at reducing violent behavior.

The analysis of "border cases" provides some interesting insights. There is a strict test score cut off for admittance to the most prestigious high school in New York. However, people that barely make the cut off seem to get into equally prestigious colleges as those who barely make the cutoff. This seems to show the high school has very little value. (However, it could also show that admissions officers favor students from a diversity of schools and may make it more difficult for those in the best school to get into their desired college.) Similar results were shown for people who got into Penn State and Harvard. Regardless of which school they chose, they seemed to have equally successful careers. (This does leave plenty of questions. Did people that went to Penn State work harder? Does Penn State have an honors program that provides a similar environment as an ivy? Is Harvard a mediocre experience for those without wealthy connections? Would people with similar academic profiles that did not apply to either show similar results?) It does seem to show that it is the person, not the circumstances that lead to success. Or perhaps that people that try and fail have a chip on their shoulder and are likely to work harder to succeed in the long run.

The author pays a debt of gratitude to Steven Levitt and Freakanomics in inspiring him to look for quirky answers to other problems. (Though he does claim that Levitt has fallen from grace to to political incorrectness and a coding issue - I guess I missed that one.) Now big data is the force that can finally put the "science" into social science. With large data sets, we can legitimately probe human behavior in a way that natural scientists can probe nature. However, there still are challenges. In some instances, "little data" can be better used. Often the best results can be found by combining multiple sources that include big data, enriched by more traditional "little data".

Monday, May 29, 2017

The Story of Music: From Babylon to the Beatles: How Music Has Shaped Civilization

Why do we have music? What was the purpose? Music has been used through much of the history of man. Ancient paintings show indicators of musical instruments. However, we have little idea what this music was. While writing and paintings from early history can be studied, we have little idea what the actual music was. There were some early attempts to provide some guidelines for the performing of music (such as the psalms.) However, these were only very basic, and did not provide enough to fully replicate the experience of the ancient musical performances. It wasn't until the last thousand years where a form of musical notation was created. Even the early notation has only reached the modern sheet music in the last few hundred years. This finally allowed music to be shared and gave rise to classical composers. However, unlike paintings that are observed in their original form, sheet music is interpreted by the performers, and remains "living" centuries after it was created.
The Story of Music traces the history of music. Famous musicians (such as Beethoven and Bach) are given their place, as are other musicians that have made contributions to the evolution of music in other ways, such as those that helped introduce chord progressions and the differences of notes. (The perfect difference between notes was dropped in favor of equal distance between notes.) Even hundreds of years ago, there was an artificial differentiation between "popular" and "artistic" music. (Ironically some of the "popular" music such as operas are now treated as "artistic".) I wish they included music in the audiobook. Descriptions of music just don't do justice to understanding of music. (Hearing comparisons of Lizt and Danny Elfman further whet my appetite for listening to the music.)
This book would have been a great candidate for an "enhanced" audiobook that included snippets of the music discussed as it was going on. I thought that somebody might have created Spotify playlists. It turns out the author has the playlists at his website: The playlists are thorough - perhaps a little too much - with many requiring over a day's worth of nonstop listening to get through.
Though the author does attempt to focus on the entire spectrum of music, the focus drifts towards what we would call "western classical music". Popular music outside the western canon does get mentioned, but not with the detail of influential classical music. There is a sense of nostalgia given for the days when the "artistic" music of the day was also the most popular. (However, there still seemed to be the acknowledgement that serious music was for the elite in days past, even if it did have popular appeal.)
The advent of music broadcasting and playback equipment finally made popular music something that could not be ignored. The performing artist became as important (or even more important) than the composer. Popular music changed some of the common tropes of music composition, but stayed fairly consistent. Composition as a popular art form became mostly confined to film music. Popular music had its own cross-pollination, with negro spirituals borrowing from the British working-class and then morphing into jazz, and then rock and roll. The book felt like it ended too soon, but that may be due to recency bias. I was hoping to hear more of how music is evolving today.

Monday, May 22, 2017

Competing Against Luck: The Story of Innovation and Customer Choice

Why would you hire a milkshake? It may be because you want something to easily eat and occupy you on a morning commute. Or, you may want to "give in" to a child and spend some time together. In one case, the milkshake is a better alternative to a donut. In another, it is competing with a toy store. Both are very different reasons for hiring. Understanding the purpose that the milkshake is fulfilling helps to better serve the customers and sell more milkshakes.

Competing Against Luck primarily uses anecdotes from various organizations to show how the jobs theory can help in making decisions about innovating. Successful new products and services help perform "jobs" that people need. These jobs may not be easily apparent in standard market research. Sometimes, there may be other factors in play that were not evident when looking at the original problem. (For example, high tech solutions may help doctor's complete activities in treating their patients. However, the emotional connection is even more important, and the technology gets in the way of their role of relating to the patient.) Often, people will not know they have a need for something new and may reject the thought of it. (American Girls dolls were panned in the initial market research, but went on to become hugely successful.)

"Objective" data can be a tricky thing, especially for ongoing operations. Everyone can use data to tell their story. However, behind the data are a series of qualitative judgement. Somebody chose what data to collect and how to measure it. People often feel they are getting "the facts", but they are really just getting numbers that were generated based on a number of opinions. (The numerical "facts" do give an air of authority even though they are often no better than opinionated judgments.)

Jobs theory can be summarized as "know the deeper meaning of why something is being done." Problems can arise when companies look at the surface reason and miss out on the true actions. Railroads faltered as they thought people were hiring them because they wanted rail travel. In fact, people just wanted a mechanism to move from point A to B. When better alternatives were available, they migrated away from rail. Similarly, there are many times where people (or individuals) don't realize the true purpose for which things are done. A job to be done is typically described in nouns and verbs. It is also provided in a general sense that can be replaced by something outside the current industry. The "job to be done" can help to explore the deeper reason for why things are done throughout life - and can also help explain some of the failures and successes we see in this world.

Wednesday, May 17, 2017

The Etymologicon: A Circular Stroll Through the Hidden Connections of the English Language

The pool in "Gene Pool", "Pooling resource" and similar usages is not related to a "Swimming pool" at all, but instead comes from the French word for chicken. Etymologicon has an exploitative, stream of conscious style that works perfectly for the subject matter. The author takes some words and gradually moves through their evolution, and in the process runs into some other interesting words. These are explored and then related to something else. Each individual group of words is an interesting read on its own. The masterwork, however, is being able to relate them all together. He also takes time to delve into some linguistic influenced history. (The Anglo-Saxon takeover of the Celt lands seemed to be an all-out attack - or peaceful coexistence. Indo-Europeans spread out in many directions with their language.) The history leaves some interesting names that sometimes are merely multiple versions of the same word in different languages. (Some placenames would be translated "Hill Hill Hill".) Neverland comes from Peter Pan by way of a part of Australia where the blacks and whites never had contact. The Starbucks name came from Moby Dick. Melville adopted a common name that came about from the Viking invasion before undergoing a number of spelling changes. Some words have changed their meaning over time. Gymnastics comes from the Greek meaning "to exercise naked" It has evolved into a specific type of athletic activity that is now done with clothes on.
There are many other great etymologies in this well written book. I would love for the author to write further follow ups.