Converting Daily Temperature Data to Sound with TwoTone

I was distracted yesterday. I saw a tweet by Alberto Cairo (@albertocairo) about a new web-based tool to convert data to sound. I have thought a lot about 'visualization' in sound since my days doing electrophysiology of locust brains at Caltech (I was even acknowledged for 'helpful comments' in a Nature paper). The tool, 'Two Tone' was "made by Datavized Technologies with support from Google News Initiative".

My masterpiece is here: Chicago vs. Redwood City (temperatures, 2015, SoundCloud)

I wanted to try it, and the first thoughts I had were to use data from a fitness tracker (e.g. steps per day), but I don't have one. Next I thought to nerdily plot my Github contributions over time. But for now I've done something a little more conventional: plot the minimum temperature over a year in Chicago, and in the Bay Area.

I started by getting data from Google's BigQuery accessing NOAA's Global Historical Climate Network (GHCN). After a little trial and error based on Google's existing examples, I figured out that the daily minimum temperature in Chicago for 2015 can be found with:

SELECT
  wx.date,
  wx.value/10.0 AS min_temperature
FROM
  `bigquery-public-data.ghcn_d.ghcnd_2015` AS wx
WHERE
  id = 'USW00094846'
  AND qflag IS NULL
  AND element = 'TMIN'
ORDER BY
  wx.date

To do the same for Redwood City, CA (the nearest station to me), I used

id = 'USC00047339'

(found here).

Lots of caveats: TwoTone probably normalizes the dynamic range, so we don't get a fair comparison of the two cities. I'd also like more control over features of the audio: the maximum tempo is 300 bpm, but audio data can be processed much faster by the brain; I'd also like to be able to combine tracks with an eye to meaningfully comparing the data. I did this in Garage Band, but by then, the data had been converted to audio waves. One of the great features of TwoTone is to manipulate sound visualizations, while still looking at the source data. I look forward to the evolution of this tool, and generally to the field of audio 'visualizations' of data.

Pope Shows the Way to Modernity in "Call to Holiness"

Pope Francis published a remarkable document today, called the "Gaudete et exsultate", an exhortation to holiness. He says its goal is to "repropose the call to holiness in a practical way for our own time". The philosophy of the document is bold and modern.

 Each of its 177 numbered paragraphs, in five named chapters, contains an expression of wisdom and a practical view on what it is to live a holy life. It is very much worth reading and meditating upon, whatever your faith.

The document resonates on many levels. It calls for acting with "Joy and a Sense of Humor", for "Going Against the Flow", and importantly, emphasizes that charity is the highest virtue. A summary in the New York Times highlights the teaching, found in paragraph 101, that caring for the poor and immigrant are as holy as the opposition to abortion:

...Our defence of the innocent unborn, for example, needs to be clear, firm and passionate, for at stake is the dignity of a human life, which is always sacred and demands love for each person, regardless of his or her stage of development. Equally sacred, however, are the lives of the poor, those already born, the destitute, the abandoned and the underprivileged, the vulnerable infirm and elderly exposed to covert euthanasia, the victims of human trafficking, new forms of slavery, and every form of rejection.[84] We cannot uphold an ideal of holiness that would ignore injustice in a world where some revel, spend with abandon and live only for the latest consumer goods, even as others look on from afar, living their entire lives in abject poverty.

It is also refreshingly modern in its form and format. It is published in HTML, with a clear and consistent structure (numbered paragraphs, chapters). Its references all have hyperlinks to the footnotes, and the footnotes themselves are hyperlinked to original sources!

At the top of the document is a list of social media links (FB, Twitter, Google +, email...), as well as a link to print and pdf for old-school applications. It also has been translated into many languages and has links to those versions at the top.

It even has breadcrumbs placing this document in context of the other documents on the website.

It comes in a mobile version, and the document redirects to the correct format for your device: http://m.vatican.va/content/francescomobile/en/apost_exhortations/documents/papa-francesco_esortazione-ap_20180319_gaudete-et-exsultate.html

Each of these components was clearly well thought-out, and underscores, subtly, Pope Francis' call to modernity. While he warns us not to be "caught up in superficial information, instant communication and virtual reality", he does not reject technology itself, and indeed has published an (almost) thoroughly modern document. The one step that is missing is to provide a hyperlinkable structure for the document. If each numbered paragraph had an id, for example, it would be possible to link directly to the paragraph, like : http://m.vatican.va/content/francescomobile/en/apost_exhortations/documents/papa-francesco_esortazione-ap_20180319_gaudete-et-exsultate.html#p101

If the Vatican is interested in taking this additional step, I would be very happy to consult on how to convert this and other papal declarations into fully standardized Akoma Ntoso.

XML Editing is Hard

I was reminded this week that our best customers at Xcential are those who try to build a legislative data system themselves first. And today's XCKD perfectly sums up why:

https://imgs.xkcd.com/comics/here_to_help.png

Git for Law Revisited

Laws change. Each time a new U.S. law is enacted, it enters a backdrop of approximately 22 million words of existing law. The new law may strike some text, add some text, and make other adjustments that trickle through the legal corpus. Seeing these changes in context would help lawmakers and the public better understand their impact.

To software engineers, this problem sounds like a perfect application for automated change management. Input an amendment, output tracked changes (see sample below). In the ideal system such changes could be seen as soon as the law is enacted -- or even while a bill is being debated. We are now much closer to this ideal.

Changes to 16 U.S.C. 3835 by law 113-79

On Quora, on this blog, and elsewhere, I've discussed some of the challenges to using git, an automated change management system, to track laws. The biggest technical challenge has been that most laws, and most amendments to those laws, have not been structured in a computer friendly way. But that is changing.

The Law Revision Counsel (LRC) compiles the U.S. Code, through careful analysis of new laws, identifying the parts of existing law that will be changed (in a process called Classification), and making those changes by hand. The drafting and revision process takes great skill and legal expertise.

So, for example, the LRC makes changes to the current U.S. Code, following the language of a law such as this one:

Sample provision, 113-79 section 2006(a)

LRC attorneys identify the affected provisions of the U.S. Code and then carry out each of these instructions (strike "The Secretary", insert "During fiscal year"..."). Since 2011, the LRC is using and publishing the final result of this analysis in XML format. One of the consequences of this format change is that it becomes feasible to automatically match the "before" to the "after" text, and produce a redlined version as seen above, showing the changes in context.

To produce this redlined version, I ran xml_diff, an open-source program written by Joshua Tauberer of govtrack.us, who also works with my company, Xcential, on modernization projects for the U.S. House. The results can be remarkably accurate. As a pre-requisite, it is necessary to have a "before" and "after" version in XML format and a small enough stretch of text to make the comparison manageable.

Automating this analysis is in its infancy, and won't (yet) work for every law. However, the progress that has been made points the way toward a future when such redlining can be shown in real-time for laws around the world.