Hello and Happy Holidays from everyone at BuzzData!
The new year is fast approaching, but we couldn’t help but push a few more new features to the site before 2011 officially comes to an end. Without further ado, here are the latest and greatest improvements we’d like you to know about:
Dataset and user profile badges to add to your websites
We have just released a set of customizable publishing badges you can now embed into other website(s), thus instantly increasing the visibility and reach of the data you generate.
Whether you want to use them on a personal blog or a company homepage, BuzzData badges make it easier than ever to share your data with the world.
Now sharing data with your social circle is as easy as a single mouse click. Want your social media followers and friends to be able to download cool data you find on BuzzData? Get the data into the Twittersphere in an instant.
Community Tasks
This is an early-stage community feature we’re quite excited about. Now whenever you create a new dataset on BuzzData, you can write in tasks that you’d like help with from others in the dataset’s Overview tab.
So what? Well, if your dataset is public, it will show up in our new global “Tasks” webpage, located just to the left of the BuzzData search box, along with the tasks you need help with.
The Tasks page is where users can peruse unfinished data projects from around the world that they can contribute to, thus helping the data community work together to achieve their goals.
To illustrate, when creating a dataset:
The global Tasks page is an early-stage feature that will continue to evolve in 2012. We hope you make use of it often and let us know how we can make it better by emailing us at support@buzzdata.com.
Now accepting direct image uploads and video URLs
Have you made a visualization of your dataset on your desktop and want to upload an image or video of it in action? No problem, you can now upload image files directly to the site and view them in our new visualization viewer.
In addition, BuzzData’s visualization viewer now allows you to post and stream videos from a variety of popular content providers such as Youtube, Vimeo and BrightCove, so you can showcase videos of interactive visualizations and other media on BuzzData as well.
Alright that’s it for now! We hope you have a lovely, stress-free winter break and a fun-filled New Years’ Eve!
By Clay Heaton, data management specialist and co-founder of The Perihelion Group. In 2010, Heaton was dispatched to a field hospital in Fond Parisien, Haiti, where he built a simple Rails-based EMR system so emergency response staff and volunteers could track medical information, triage and treatment.
I’m a public health professional who specializes in data management and the humanitarian response to crises. It’s a field where timely and organized data-sharing positively impacts large groups of vulnerable people. I hope for a day when people who aren’t data experts have a go-to place for creating nimble data repositories others can access, query and update in real time, immediately following disasters like the recent ones in Japan and Haiti.
Field-based data management is a challenge because first-response teams focus almost exclusively on search, rescue and medical missions. They lack access to a network, and have little time (or patience) to learn new tools while surrounded by catastrophe. Excel is the go-to application for immediate data tracking needs, and, in the hands of the right person, holds up well in difficult conditions.
However, humanitarian organizations frequently rotate their staff in and out of the field to avoid extensive fatigue. Consequently, spreadsheets created during the early hours of response efforts are usually passed around. Each subsequent staff member and volunteer has their own idea about how the data should be structured, what should be tracked, and how important it is to collect data at all.
This lack of consistency in data workflow can cause major problems: A single inaccuracy in a spreadsheet can lead to dozens of hours lost trying to locate an unaccompanied child who supposedly is in the hospital, but actually has been discharged already. Worse, such mistakes can have terrifying legal implications for well-intentioned volunteers and staff in the field.A physician on site in Haiti holds a copy of a medical record that came out of the EMR app built by Clay Heaton. It was the first medical record the patient ever had received. Photo by Clay Heaton.
Staffing logistics aren’t the only challenge, however. When you’re trying to manage the health and whereabouts of hundreds of patients and staff and you have limited (or no) Internet access, web-based solutions can be problematic. Given an Internet connection, Google Spreadsheets has been the best off-the-shelf tool that I have seen so far. But disagreements regarding sheet structure, coupled with rotating staff, often leads to unregulated cloning of datasets. It is all too common to see colleagues at the same field location working off of two different copies of the same spreadsheet, unaware that they aren’t sharing the same data.
Companies such as Google and groups like Crisis Commons work from headquarters locations, building grand tools that help reunite families and give field response teams an outlet for requesting supplies. But the single largest data management challenge during emergency response is not a grand “30,000-foot” problem. It is what I call the three-foot problem.
Medical supplies in the Fond Parisien Hospital. Photo by Clay Heaton.
Given the best tools in the world, web-based, standalone, mobile, or otherwise, the biggest data challenge is informing responders of the existence of the tool, training them on it, and finally, giving them the incentive to actually use it. While professional staff are far more likely to use the tools than volunteer staff, volunteers often make up the vast majority of field-based responders. The American Red Cross volunteer-to-staff ratio, for instance, is on the order of 40:1.
Training and encouraging volunteers to use data management tools during a crisis is extremely difficult. Many people volunteer with an adrenaline-based attitude that they are going to save the sick and weary. To those people, time spent entering data into a computer or telephone is equivalent to lives surrendered.
I often deploy with field teams to build on-site data management tools, in an effort to keep response efforts consolidated and organized. I use the tools available to me, from Excel to Filemaker Pro, MS Access, or Open Office. I prefer to just roll a simple Rails app, however, because I can run it off of an ad-hoc network, anybody with a laptop or smart phone can access it (within range of the network), it doesn’t require commercial software, and most people are familiar with the use of a web browser.
In Haiti, for example, I built an EMR system that included pharmacy and warehouse supply tracking, full patient medical records, disease surveillance tools, and a requisition wish list for supplies. I deployed the components as they were ready, beginning with the pharmacy tracking tools. I built the entire system from scratch and completed it in under a week.
We still had problems with poor data entry, volunteers who refused to use the system, a cranky generator that sent power spikes to the server, and teams of staff with 5 different native languages. You can see a small example of the data from the Rails app that I build in Haiti on my BuzzData account. While messy, it is one of the best records of medicines dispensed at a field hospital during humanitarian response efforts that ever has been recorded.
From the Red Cross to the UN to Google, even the big players in this field frequently botch data collection. Most frequently, it is because they work remotely on grand solutions, oblivious to the 3-foot problem. One large and well-respected NGO (name withheld) manages refugee camps and field hospitals with tens of thousands of residents and manage the roster with a simple Excel sheet into which they enter all demographics, medical and surgical treatment information, and contact information for relatives in the camp. Note: this is all on the same sheet, not even on multiple sheets in the same workbook.
Field hospitals, displaced persons camps, and emergency operations eventually close. The staff and volunteers that run them scatter back to the institutions, agencies, and companies whence they came. They spend a few weeks recovering and then they want to share their data, mine it for trends, and try to understand how they can improve their response efforts for the next crisis. Their data often hardly is better than garbage. Sometimes you just have to work with garbage because it’s the best that you have. This is pretty sad to a data nerd like me.
Nevertheless, field collaborators need to share their data so they can publish studies and improve response efforts. This often is done in Google Spreadsheets. But the scourge of any spreadsheet application is that it simplifies complex data relationships. Sure, you can flatten everything down into a dataset that has 500 columns, but it becomes so daunting that it’s nearly impossible to navigate.
BuzzData wasn’t built to manage emergency response data but I think it has the potential to be a good tool for people working in the field to share data and cooperatively mine it for meaning post-emergency. However, field data can be pretty complex. It does the datasets a disservice to convert them to flat files, dropping relationships and leaving the burden of reestablishing them to people doing analysis or to statistical analysis packages.
Why flatten them just to have to do the work to reestablish the relationships in other software? It’s much easier to poke around in data and explore a dataset when the relations are intact (if it was relational to begin with). This is a problem experienced in all industries. For me, taking the time to try to generate a meaningful query from a dataset and upload it to BuzzData for sharing ends up being an additional intermediate step between the data repository and the end user.
I hope that helps you to understand my perspective on BuzzData. As a data nerd, I find BuzzData exciting and I look forward to watching it grow.
This guest blog post evolved from a dialogue between one of our early users and our CTO Pete Forde. What I love most about this is the fact that the focus of our development is helping bring these critical issues to light.
