Gartner Cloud DBMS Report Names MarkLogic a Visionary

Can NoSQL Solve Polling Dilemma?

The stunning upset of Eric Cantor in the Virginia congressional race was largely stupendous because the pollsters got it so wrong – by “18 standard-deviation points” said one analyst. It was “pollster malfeasance” by the polling firm McLaughlin & Associates, and it should get the equivalent of the “pollster death penalty” said a commenter to Frank Lutz’ New York Times piece.

But to a man and a woman on the street in Virginia it was not that much of an upset; People did not like Cantor. In a similar scenario, David Cameron is up against Ed Millibrand – and one columnist wrote that political analysts can’t grasp “the sheer scale of Ed Milliband’s political incompetence and stupidity.”

Why are pollsters so tone deaf to those emotions? Well the problem right off the bat is that polls are highly structured: Do you like this or that? Are we in the right direction?: all the time, some of the time, none of the time. The speed at which polls are being taken – we are stamping our foot for a response before the last data point is entered – has created an over-reliance on structured answers.

Structured answers has made it easier to tabulate and analyze information. But only those answers that fit a specific structure. To a question of “do I prefer chocolate versus vanilla ice cream” the answer would be chocolate … unless I want to put toppings on my ice cream or unless I put ice cream on pie, or unless it is my son’s home-made vanilla that is to die for. At the end of a year I have had at least three times as much vanilla as chocolate!! (And actually more, because my son makes it every weekend.)

The narrative holds the key — and the truth. In healthcare, companies like M*Modal are helping healthcare providers analyze physician notes to determine best course of patient care. M*Modal can efficiently do this because it has created a linguistic engine that analyzes words used to determine context – and creates rich, hierarchical XML metadata that then flows into MarkLogic for search and analysis.

Could pollsters do the same? Absolutely. They could be mining comments, conducting live interviews (which they should be doing anyway but that is for someone else to write about), having conversations! The recipe is pretty simple: transcribe notes, run the text through vocabularies and thesauri so that we know: “He’s an idiot, moron, obtuse, incompetent, shill” maps to a strong negative.

Humans being what they are, no poll will ever be 100 percent accurate. But 18 standard deviation points is pretty much in the realm of flipping a coin. And with the amount of coins being spent on polling, I am fairly certain clients want better odds of gaining true insights to public opinion.

Chief Content Strategist

Responsible for overall content strategy and developing integrated content delivery systems for MarkLogic. She is a former online executive with Gannett with astute business sense, a metaphorical communication style and no fear of technology. Diane has delivered speeches to global audiences on using technologies to transform business. She believes that regardless of industry or audience, "unless the content is highly relevant -- and perceived to be valuable by the individual or organization -- it is worthless." 

Start a discussion

Connect with the community

STACK OVERFLOW

EVENTS

GITHUB COMMUNITY

Most Recent

View All

Digital Acceleration Series: Powering MDM with MarkLogic

Our next event series covers key aspects of MDM including data integration, third-party data, data governance, and data security -- and how MarkLogic brings all of these elements together in one future-facing, agile MDM data hub.
Read Article

Of Data Warehouses, Data Marts, Data Lakes … and Data Hubs

New technology solutions arise in response to new business needs. Learn why a data hub platform makes the most sense for complex data.
Read Article

5 Key Findings from MarkLogic-Sponsored Financial Data Leaders Study

Financial institutions differ in their levels of maturity in managing and utilizing their enterprise data. To understand trends and winning strategies in getting the greatest value from this data, we recently co-sponsored a survey with the Financial Information Management WBR Insights research division.
Read Article
This website uses cookies.

By continuing to use this website you are giving consent to cookies being used in accordance with the MarkLogic Privacy Statement.