so far, i've used
and Python (basic)
to report on polling issues and make election data more accessible
Read about my adventures as a data novice below, and to see some examples of what I've done, click here.
April 23, 2018
This semester, I worked with ProPublica and NYTimes reporter Derek Willis on a project he cofounded called OpenElections. Willis, an elections junkie, and his partner Serdar Tumgoren identified a major problem for journalists in search of election data: While election results are technically public records, they are often difficult to retrieve. OpenElections seeks to do that work for them and publish hard-to-get results in a machine-readable format for anyone to access. This project, funded by the Knight Foundation among many other data giants, realizes American news media’s role in protecting democracy, and it’s designed in a way that is indeed accessible to the public – especially volunteers.
I sought out this opportunity after realizing how valuable data skills are to investigative reporters. Previously, I had worked with professors Meredith Cummings and Chip Brantley on ProPublica’s Electionland project, where we used several tools (like Dataminr, Reverse Image Search, Google Maps, Facebook Live, and TweetDeck) to scan social media for polling issues during the 2016 Presidential election. That’s when a print girl began to fall in love with digital media. Since then, I’ve been looking for ways to dig deeper in my work. In my Advanced News Writing and Reporting class last fall, Cummings compiled a detailed guide to FOIA requests and always urged us to find information for ourselves. That’s when I became familiar with UA’s OIRA system and started contributing more data-driven stories to places like the Crimson White. I was unaware of any other data-specific classes the department offered, so I reached out to Brantley to see if he had anything in mind, and he connected me with Willis.
When I started this January, I knew nothing about data collection other than through social media. I also didn’t know that much about elections – and especially not about how we classify and sort election data. While working with Willis, I realized just how tedious data gathering is. At one point, he gave me a 400-page file that he wanted me to clean. If I did this successfully, he told me, I would realize why newsrooms employ people to do these things for them. Suffice it to say, I disappointed myself and was unable to copy the file into Excel. But, like all things this semester, this was a learning experience. I gained an immense appreciation for the hard work many journalists may take for granted, and I realized that I liked talking to people a lot more than crunching numbers. Despite my hang-up with that massive file, this experience taught me so many things about both data and myself. The curriculum we made required me to learn basic manual data entry, but it also introduced me to the GitHub community, taught me basic commands, and allowed me to write some code on my own. Willis and Tumgoren also gave me a lot of ownership in their project, allowing me to communicate with other users and re-envision their site. In the end, I think we were all able to learn from each other, which is something I’ve always wanted in an internship.
The highlight of my semester, however, was when Willis invited me to NICAR, an investigative reporting conference. Previously, I had been frustrated with student media, where I was a page designer yearning to do more. Because I hadn’t been introduced to data journalism until recently, and because I didn’t feel like I really had an outlet to practice these skills, I stayed put. But at NICAR, I was inspired daily, and I left with plenty of tools to start writing impactful stories. I started thinking about ways that the CW can start partnering with digital minds, and I was recently given a chance to do that as the production editor. More importantly, though, this experience made me more confident in pursuing a career in investigative work.
Installed and learned how to use Terminal, GitHub, Tabula, Homebrew, xpdf, TextWrangler, SSH key, Google Groups, Slack, HelloFax, OpenElections Tracker, and Python
Used Excel to reformat parsed NY county .pdf results into .csv files (Total = 4)
Called AL, WV, and TX county clerks to retrieve election data (Total = at least 30)
Followed up with emailed or faxed requests (Total = 19)
Wrote FOIA requests when no response (Total = 9)
Pushed cleaned data to GitHub repositories (Total = 4)
Pushed source files to GitHub repositories (Total = 17)
Completed python assignments to clean data (Total = 2)
Became a NICAR (National Institute for Computer Assisted Reporting) member
Attended national CAR conference and took notes on sessions (Total = 14)
Met professional reporters, including the two OpenElections cofounders
Drafted questions and was active on Slack to gauge user needs for potential website re-design