Gartner Cloud DBMS Report Names MarkLogic a Visionary

Santa Scales Out

Over the years, I’ve heard from sources up North about how Santa has been a data pioneer.

First, he progressed to NoSQL for more flexibility to manage his list and do his matching in record time. It turns out the Big Syndicate up North was actually an innovator!

Then his team upped his game and invested in semantics to improve his matches. Getting the gifts to the right kids meant nothing if they weren’t the right ones. In fact, when kids didn’t like their gifts, sometimes Santa would get sad. So his team upgraded the system to take into account the context of the kid.

However, transforming your business to be data-centric is never done. Recently my source at the North Pole revealed that Secret Santa, one of Santa’s early crowdsourcing projects, was in need of some help.

Spreading the spirit of Santa to families and workplaces with Secret Santa was a great idea, but the process was getting pretty antiquated. Putting names in the hat and picking them works great … as long as it’s a simple group. But like all social networks, Secret Santa is about mixing together multiple types of people and as it goes digital, scaling it out. Multiple families picking together and not getting their siblings can be complicatedand then enterprise applications like departments in a company, take it well beyond simple paper and a hat.

So, it was back to the keyboard for his programming elves. The goal was simple: start with the list of people that includes their group. Then, for each person, assign a pick that isn’t part of their group and (of course) hasn’t been picked before.

But like all transformations from physical to digital, there were some challenges, the most notable being data security. Santa was early into data, but he was also early into security, demanding that all of his projects not only be run in secure environments, but also properly enforce permissions with appropriate access.

But, my source up North told me the elves were up to the task. In fact, more than one team was assigned to it. Ever since Santa adopted NoSQL and semantics, the team has gotten used to taking multiple approaches to solving a problem. It all starts with the data, but sometimes the elves can get an advantage with some old-skool XML processing. It turns out that XML is still everywhere in enterprises. And for good reason as self-describing data has a great deal of power. But programming directly with this model isn’t for everyone, so many on the team have gravitated to Javascript.

This time around, Santa let each team have a go. And the results are pretty interesting:

  • The Old Skool team started with a simple data structure and then used some nice tricks to trim the list based on the picker’s group. But things get complicated with the already-picked list as a global variable needs to be updated with the picks. Fortunately, maps enable you to update data during the processing. Finally, the team noticed what all of those hat pickers have encounteredthis problem is a multi-pass problema given random match doesn’t always distribute the picks between groups, and it has to be redone until you get a good match.

For all the details, see the really old-skool post on the original XQuery blog.

  • Team Javascript got more ambitious and really tackled the enterprise scaling and security. Instead of just doing the processing, they proceeded to creating and managing data. This enterprise approach means that picking is resilient and secureperhaps suitable for some really big inter-departmental pickingbut it was also a bit tougher to deliver

      Read all about this robust and accessible approach.

Once again, Santa is ahead of the curve, enabling groups to automate their Secret Santa and scaling out his crowdsourcing to a new generation!

Matt Turner - Chief Technology Officer - Media Publishing | MarkLogic

Matt Turner is the CTO, Media and Manufacturing at MarkLogic where he develops strategy and solutions for the media, entertainment and manufacturing markets. Matt works with customers and prospects to develop MarkLogic enterprise NoSQL operational data hubs that enable them to get the most of their data and deliver their products to the fans, audiences and customers that love them.

Before joining MarkLogic, Matt was at Sony Music and PC World developing innovative information and content delivery applications.

Start a discussion

Connect with the community

STACK OVERFLOW

EVENTS

GITHUB COMMUNITY

Most Recent

View All

Digital Acceleration Series: Powering MDM with MarkLogic

Our next event series covers key aspects of MDM including data integration, third-party data, data governance, and data security -- and how MarkLogic brings all of these elements together in one future-facing, agile MDM data hub.
Read Article

Of Data Warehouses, Data Marts, Data Lakes … and Data Hubs

New technology solutions arise in response to new business needs. Learn why a data hub platform makes the most sense for complex data.
Read Article

5 Key Findings from MarkLogic-Sponsored Financial Data Leaders Study

Financial institutions differ in their levels of maturity in managing and utilizing their enterprise data. To understand trends and winning strategies in getting the greatest value from this data, we recently co-sponsored a survey with the Financial Information Management WBR Insights research division.
Read Article
This website uses cookies.

By continuing to use this website you are giving consent to cookies being used in accordance with the MarkLogic Privacy Statement.