Integrating Magpi and DHIS2 with OpenFn

Guest blog by: Taylor Downs, Founder of OpenFn.org

JXXm1-JZAbout two weeks ago I had the pleasure of working with the team at Magpi on some changes to their API that would allow them to connect Magpi with OpenFn—an enterprise-grade integration platform that’s been designed for the international development community. Joel and I thought this would be a great opportunity to discuss how Magpi can be used in conjunction with other technologies in the ICT4D space to create powerful, integrated systems that get data into the hands of those who need it, fast.

I know that I don’t need to explain Magpi to this audience, but thinking of Magpi as part of a larger system that works for the entire organization may be new to some of you. Many organizations in the public health space are familiar with DHIS2. This is just one of many management information systems (like Salesforce, IBM Maximo, and many more) that help businesses, non-profits, and governments drive efficiency, measure and adjust effectiveness, and ultimately, better deliver their products and services.

Today, I want to take a look at how a Magpi-to-DHIS2 integration can help an organization become more data driven.

dhis-laptop

Integrating Magpi with DHIS2

dhis2When Magpi data is captured out in the field, it syncs to Magpi.com as soon as you’re back in network coverage. Logging into Magpi gives managers quick web-access to the information that was gathered on mobile phones. That survey data may, however, form part of a larger public health picture involving district, provincial, and national health and population data on DHIS2 — one that involves more stakeholders, new dashboards, and visualizations or reports essential to the planning, implementation, and assessment of key public health projects. Real-time data in Magpi is powerful, but getting Magpi data into DHIS2 in real-time can be a game-changer.

Using Open Function, organizations can automate this Magpi-to-DHIS2 workflow so that human beings no longer need to export data, reformat it in Excel, and perform tedious manual uploads to DHIS2. They can save time, save money, prevent data processing errors, and free staff from a truly boring job. (At OpenFn, we‘re known to sometimes actually shout, “Don’t make humans do stuff that computers can!”)

Fetch new data from Magpi using Open Function

After creating an account at OpenFn.org/signup users need to set up a credential so that they can access their Magpi data. The key piece of info you’ll need from Magpi is an “Access Token” and you can find that by clicking on your username in Magpi and selecting “API” from the menu.

Finding your Magpi access token

Finding your Magpi access token

Creating a Magpi credential on OpenFn

Creating a Magpi credential on OpenFn

 

After setting up your credential for Magpi, you’ll need to give OpenFn it’s first “trigger” and “job”. A job is a specific task that OpenFn performs for you whenever something happens. A job might get triggered by the arrival of some new data in your OpenFn “inbox” or it might simply get triggered by the passage of time. We’ll use a “timer trigger” to fetch data from Magpi every 20 seconds.

Click on Triggers, select New, and add a 20 second timer trigger.

Now click on Jobs, select New, and choose your credential, trigger, and the Magpi adaptor. Writing a Magpi fetch job is as simple as pasting the 5 lines below and replacing your surveyId and postURL:

fetchSurveyData({
 “surveyId”: 37479,
 “afterDate”: “2016–01–01”,
 “postUrl”: “https://www.openfn.org/inbox/your-secret-code"
});

This is what my first Magpi job looks like:

Creating a job that fetches new survey data from Magpi

Creating a job that fetches new survey data from Magpi

 

 

When you turn on the job you’ll start seeing Magpi data show up in your OpenFn inbox in the first 20 seconds. Now you need to decide how to process that data in DHIS2. For this example, I’m imagining that Magpi surveys were used to take attendance at community health worker events so I’d like new survey submissions to create new events in DHIS2.

Sending Magpi data to DHIS2

Just like your Magpi credential, you’ll now need to create a DHIS2 credential.

You’ll also need to set up a trigger, but instead of using a timer trigger you’ll create a “message filter” trigger that waits for new Magpi data to arrive and then kicks of the DHIS2 job. Message filter triggers are just like filters in Gmail. Pick a part of your Magpi survey data to match like the surveyId:

Creating a message filter trigger for Magpi data

Creating a message filter trigger for Magpi data

Now you can go ahead and set up your DHIS2 job. Choose your DHIS2 credential, the Magpi data trigger you’ve just created, and the DHIS2 adaptor.

When writing a job for DHIS2, we’re going to use the “event” function and we’ll need to identify some standard information like the “program”, “orgUnit”, and “eventDate”. We’ll also need to pass a dataValues object to DHIS2. What that dataValues object includes will vary depending on what data you’re capturing in DHIS2. There is fantastic documentation on the DHIS2 Events API at docs.dhis2.org. So, this is what a DHIS2 job could look like. We’re setting a few attributes with static text, and others are being pulled dynamically from our incoming Magpi data using the dataValue(path) function.

 
Creating a job that inserts a new DHIS2 event with Magpi survey data.

Creating a job that inserts a new DHIS2 event with Magpi survey data.

 

When the DHIS2 job is switched on, it will run automatically whenever data arrives from Magpi. Data will be loaded into your DHIS2 instance, and you’ll be able to monitor the connection from your “Run History” page:

Viewing run history for your Magpi — DHIS2 integration.

Viewing run history for your Magpi — DHIS2 integration.

Since many of OpenFn’s users work with mission critical or sensitive data, we take logging and error-handling seriously. You’ll be able to view and even re-process past runs on the platform or subscribe to instant email notifications in case any of the data isn’t able to reach DHIS2. (This could happen because new required fields have been added, your DHIS2 service becomes unavailable, or changes have been made to the DHIS2 api.)

When working with larger teams to build integrations across disparate systems, you’ll want to make use of OpenFn’s GitHub connection to manage all of the changes to your integration jobs using the leading version control software on the web. You can link particular jobs to files in GitHub repositories so that they’re automatically updated by commits to a particular branch.


Some final thoughts

Stepping back from the technical aspects of these integrations, we should remember that the real work involved in building a data driven organization requires precisely zero lines of code.

if (doubts) {

return “Getting the people & processes right is harder!"

};

I’ve written a number of posts on the less technical aspects of integration and I’d encourage you to take a look at both Roger Federated Architecturesand The Power of Crow’s Feet.

Be it supply chain integrations with Salesforce, work-order management with IBM Maximo, or the public health systems running on DHIS2, I’m incredibly excited to see how this new adaptor gets used. If you’ve got ideas, please share them, and if you need any help setting up OpenFn for your organization, don’t hesitate to reach out.


If you would like to tell our audience about your use of Magpi, we'd be happy to
sponsor your "guest blog". Please contact us at info@magpi.com

This article also published on Medium

LINKAGES Mozambique HIV Services Report

The LINKAGES project aims to accelerate the ability of governments, organizations working with key populations at risk of HIV, and private-sector providers to plan and implement services that reduce HIV transmission among key populations and their sexual partners and extend the lives of those already living with HIV.

Learn more in the full report.

Download Report