Dec 9, 2011

Innovation happens at the Intersections

On December 3rd I attended part of the worldwide OpenData Hackathon, locally in Riga, Latvia.  A few dozen people were there and it was a good mix -- idea people and political activists, data analysts, and coders/hackers.  Will this mix of people and interests create a nice mix of ideas and outcomes?

Our first data set were the donations in 2010 and 2011 to various Latvian political parties from various individuals and companies.  After filtering out the smallest donations, we saw a familiar pattern -- most donors choose one party, and one party only, to donate to.  Several donors contributed to multiple parties -- spreading their bets (or "investments").  The donation pattern created a mostly hub-and-spoke network which is below. The green nodes are the parties, and the black nodes are the donors.  A black node is linked to a green node if a contribution above a certain amount was made.  Kind of a nice network for the holidays, eh?

Next we dug into the voting patterns of the Latvian parliament, the Saeima.  An early comment was that the data set would probably not show anything interesting — afterall, the parties keep a tight reign on the deputies, strongly encouraging everyone to vote similarly.  Being a long time student of social networks and emergent organization, I was not so sure.  I said, "Let's give a try and let the data speak"

Even though the data/results of each deputy vote is published on the Saeima web site, the data was not easy to extract for pattern-matching.  Raimonds Simanovskis and Jānis Baiza made a heroic effort and got the voting record data of the 11th Saeima.  Uldis Bojars ran the data through Python scripts to give us network data.

Our first few attempts at interesting visualizations were not successful.  Unfortunately the data contained many counts from procedural votes — where all deputies usually vote in the affirmative to move legislation along.  We had to filter the procedural votes out so that only the votes on substantial legislation remained. Once that was accomplished we were left with data that would expose voting patterns on important issues.  Like often in social network analysis, those who thought they knew the data, were surprised.  The deputies did not always vote as their parties instructed!  Some interesting political bedfellows emerged.  What was even more interesting where the patterns of the individual parties and where they end up on the network map.

We filtered the data to show the stronger, higher occurrence patterns.  The first pattern we saw was the Latvian/Russian split in the parliament.  The largely Russian Harmony Centre party was a an obvious cluster — they are the opposition party. The ruling coalition of Unity, Zatler Reform Party, and the National Alliance —all majority ethnic Latvian — formed an integrated cluster.  The remaining party, the Greens/Farmers had a choice — 1)isolation, 2)joining one of the two clusters, or 3)bridging the two clusters.  They have decided to be the bridge in the early months of the 11th Saeima.  

The network map of deputy voting patterns is below.  Two deputies are connected if they voted the same on many important pieces of legislation.  Harmony Centre are the bright red nodes on the right, the dark green nodes in the middle are the Greens/Farmers.  The darker maroon nodes are the National Alliance, the lighter purple nodes are Zatler's Reform Party and the light green nodes are Unity.  They are all colored by their branding colors.  The number of nodes shown does not add up to the total number of deputies in parliament — some of the deputies did not survive the high cutoff in this data set for number of votes.

The network map above shows the ruling coalition on the left side, with the near opposition in the center, and the far opposition on the right.  The ruling coalition and the far opposition do not have a pair of deputies voting alike.  The between party -- Greens/Farmers -- appear to be keeping the network from totally fragmenting.

We will allow the political experts in Latvia to explain the patterns in the above network — we can provide a copy of the network map with all of the deputy names.  How will this voting pattern affect a small country coming out of a deep economic recession? I for one, am happy not to see five disconnected clusters.  The bridging ties in the above network give me hope we can work toward a better Latvia for all.

This day of open data exploration was a perfect example of how innovation happens at the intersections!  People of different skills, perspectives, knowledge and goals came together around open government data and at the end of the day had formed an emergent network that was connected and moving forward, yet was not subverting individual talents and goals.  Like I always say:  Connect on your similarities and benefit from your differences!  

Maybe this mantra will play out in the Saeima also?

No comments: