We’ve joined forces with Smartlogic to reveal smarter decisions—together.

MarkLogic World Hackathon: Building a Data Hub in the Cloud

Building on last year’s momentum, we held our annual hackathon again at MarkLogic World 2019, the annual conference for MarkLogic users.

Held in the conference’s Developer Lounge, conference attendees met after two days of information overflow and buckled down to show off what they could do with a MarkLogic Data Hub. Teamed up with strangers, Hackathon participants got their hands dirty by extending a data hub and deploying their application to the new MarkLogic Data Hub Service, while learning a ton along the way. We wrapped up late in the evening with presentations, judging, and prizes. Missed it? There’s always next year!

The Challenge

We gave teams an existing data hub with data about Fortune 500 companies and products and social media feeds. The teams got creative by further enhancing the data and digging deeper into data hub features. In front of their peers, teams demonstrated their solutions through a REST extension hosted on the MarkLogic Data Hub Service.

We placed participants onto teams of three, separating colleagues so participants could meet new people. We also distributed MarkLogic expertise so those with more experience teamed up with folks new to the product. Then, we paired teams with a MarkLogic mentor who would guide them through brainstorming, collaboration, and troubleshooting. There were MarkLogic representatives floating around the rooms, ready to help and answer questions at the raise of a hand.

Some of the participants never worked with MarkLogic before, and many were new to the idea of a data hub. And what a better way to learn than by sitting side-by-side with the folks responsible for engineering it?

“It’s a great opportunity to get some hands-on practice with new features while meeting other members of the developer community” said David Cassel, MarkLogic Developer and Architect at 4V Services, LLC.

Curious what the participants came up with?

And the Winner Is…

Team SHK, comprised of Stephen Brooks from Avalon Consulting, LLC, Hannah Sporer from Selective Insurance, and Koushik Chatterjee from Deloitte, took home the coveted first place prize. They tackled the hackathon challenge head-on, further enriching the Twitter data they were provided with sentiment analysis. They used the third-party MeaningCloud API to determine if the tweets were generally positive or negative. They gave a demonstration of their working custom REST endpoint deployed to the MarkLogic Data Hub Service.

Team SHK built a data service that served two purposes. First, given a location input, the service returns nearby companies and sentiments about that company from local social media users. This answered several business questions. Are people generally happy about Company X nearby? Would this be a good area to expand into or release a new product into? Second, given a company, the service provides sentiments from tweets about that company and the corresponding locations. In other words, where is Company Y successful with user interactions?

Team SHK Data Service Functionality

Figure 1: Given location or company inputs, Team SHK’s data service returned relevant information about companies nearby and sentiment analysis of the tweets.

Team SHK Data Service Sentiment Analysis

Figure 2: Given a tweet, the MeaningCloud API returns sentiment analysis.

Team SHK Data Hub Harmonize Flow

Figure 3: Within the data hub, a harmonize flow combines the twitter data with the sentiment analysis.

Given more time, Team SHK would have enriched their data hub with weather-related data, harmonizing the local weather at the time of the tweet, to filter sentiments based on weather and determine the weather when comments were made. One of their goals was to produce actionable business data, such as determining what time of year and in what weather conditions did a product became popular. For example, if a drink manufacturer found that a particular beverage was really popular on hot, humid, summer days, sunny scenes may be particularly relevant in advertisements for that beverage. In addition, Team SHK would have liked to implement a date range restriction so that the data service would return results within a date range.

They won based on their creative solutions in addressing business needs. In particular, they used MarkLogic 9 features and Data Hub concepts and functionality to create a data service, and of course, deployed the hub and service to the MarkLogic Data Hub Service.  Great job, Team SHK!

Team SHK - Winners of MLW Hackathon 2019

Figure 4: Team SHK (from left to right) with their MarkLogic mentor Ruth Stryker far left: Stephen Brooks, Koushik Chatterjee, and Hannah Sporer.

We also want to give a shout out to the second place team, Team Craft Data Hub, which included Gabriel Aviles, Orbis Technologies, Inc.; David Cassel, 4V Services; and Bruce Hecht, IEEE. The team came up with the concept for “a craft beer data hub” after enjoying a few at the Hackathon! They wanted to answer questions about a particular beer of choice, such as: what company made this beer, and what other products do they produce? They augmented the data given with Fortune 1000 data and deployed to the MarkLogic Data Hub Service as well. Pretty impressive!

We’re already thinking about MarkLogic World 2020 and how to make the Hackathon experience an even better one. As always, our goal is to combine learning with a fun environment. One of the members of the winning team told us after: “It was great having coaches and others to help answer our questions.  Before the hackathon, I did not realize how straight forward it was to enrich data using microservices or online APIs.  I think this is a powerful feature that I’ll use again in the future.”

Are you up for the challenge?

Want to try your hand at the challenge?

Download the tech:

Take the courses to get up to speed:

And see what you come up with! Want to deploy to the MarkLogic Data Hub Service too? Go for it!

Trinh Lieu is the Developer Community Manager at MarkLogic. Her career has spanned a variety of roles in software including development, QA, architecture, administration, operations, and training. Trinh enjoys learning about new technologies and meeting the people who work with them.

Start a discussion

Connect with the community




Most Recent

View All

The Power of Abstractions

Learn about the use of metadata to abstract data - and create data agility - to enable organizational learning in complex data domains.
Read Article

Pharma R&D Has an Interesting Problem

A new, external threat - like IP leakage - can force modernization of an application landscape that really needed it anyway. Learn how it works.
Read Article

Digital Acceleration Series: Powering MDM with MarkLogic

Our next event series covers key aspects of MDM including data integration, third-party data, data governance, and data security -- and how MarkLogic brings all of these elements together in one future-facing, agile MDM data hub.
Read Article
This website uses cookies.

By continuing to use this website you are giving consent to cookies being used in accordance with the MarkLogic Privacy Statement.