The Editor’s New Assistant- Algorithms

Journalism emerged to address the problem of information scarcity in the beginning of the 20th century. Now, the opposite problem is emerging and taking shape- the problem of information abundance. As AppliedXL’s founder, Francesco Marconi tweeted- “It’s more difficult to filter information than it is to create it.”  

As a civilization, we produce and store more information than ever in human history. A World Economic Forum report suggested that by 2025, an estimated 463 exabytes of data will be created each day globally – that’s the equivalent of 212,765,957 DVDs per day! Thankfully we don’t use DVDs anymore- but that is not necessarily the concern here. How are we to filter out unwanted information and retain and propagate valuable data is a concern plaguing every industry, chiefly among them – Journalism. The resolution to this growing problem is artificial intelligence (no surprise here), particularly machine learning and natural language processing. We built a quick list of projects that are building capabilities to evaluate data for ‘newsworthiness’ and to integrate computational journalists to ground reporting on facts.   

The Associated Press started its efforts back in 2013. Currently, they use natural language generation to automate part of AP’s financial and sports coverage- the effect has been in the scale of production. In the business section, they went from covering 300 companies with human journalists to covering over 4,400 companies. Even though the coverage volume increased significantly, the error rate went down. Now, that is speed with accuracy.  

The Wall Street Journal’s Talk 2020 is an AI-powered text analysis platform and search tool that helps the newsroom to access 30 years of public statements made by politicians. Rapidly processing transcripts from speeches, rallies, interviews and press conferences, the tool helps reporters track what candidates are saying, investigate their stances in the past, explore speech patterns and perform other text analyses. What started as a project meant for internal use is now available to subscribers.  

Another success story was Washington Post’s Heliograf. Their in-house automated storytelling technology-enabled The Post to create a new model for hyperlocal coverage. They were able to pay attention to weekly Washington, D.C.-area high school football games with the same focus they gave to the congressional elections.   

Bloomberg introduced ‘Cyborg’ to scrape through complex earning reports to produce headlines and a bullet-pointed story instantly. This used to take a regular reporter roughly 30 mins, to say the least. The media giant also uses AI to scan global social media quickly and at scale to spot newsworthy events. Automation at the moment touches 30% of all Bloomberg content.  

Then, of course, we also have the friendly- Voitto. Finland’s national broadcaster Yle, created Voitto to work as a robot journalist with the ability to churn out over 100 articles and 250 visualizations a week, and as a smart news assistant. As the latter, Voitto lives in mobile devices to recommend users with interesting and personalized news content. It uses machine learning to improve its recommendations by learning from the user’s reading history, their interactions on their lock screen and also direct feedback.  

All major newsrooms globally are building great AI tools to bring in more transparency and enhance relevancy. Unfortunately, this seems to be a lopsided development, with most emerging and small newsrooms finding the inevitable AI-led revolution hard to integrate or grasp in its entirety.   

Charlie Beckett, founder of LSE’s think tank- Polis, decisively concludes in his report on AI and journalism: “We are at another critical historical moment. If we value journalism as a social good, provided by humans for humans, then we have a window of perhaps 2-5 years, when news organisations must get across this technology.” At Meradesh, we agree with the urgency of the situation. We made it our mission to support bold and forward-thinking media houses in their quest to achieve scale and credibility using AI-led technologies.  

We are here to support you in scaling your organization and building innovative media products.