A note from the CEO on Stears Elections

From our release of Nigeria’s first predictive electoral poll on 7th February 2023, Stears has played an outsized role in making data an integral part of Nigerian elections. What started as an experiment in 2019, when we built the first real-time election database, has inspired a wave of monitoring platforms expanding on our work and bringing data to the forefront of elections.

Over the past few days, we have been humbled that millions worldwide have turned to Stears for election information. And we are proud that while pursuing our goal of building the world’s most trusted source of African data, we have changed how businesses approach elections and energised public participation.

However, success is a double-edged sword. It is exciting to pioneer such an effort, but with innovation comes its challenges.

Between Saturday, 25th February 2023 and Wednesday, 29th February 2023, Stears’ platforms were overwhelmed by the huge demand for data on Nigeria’s 2023 Presidential and National Assembly Elections, leading to intermittent downtime for many users.

During this period, some users experienced delays while accessing our website and mobile app, error messages when attempting to log into the product and overall latency with our features. In some cases, users needed clarification about how to interpret our data. 

We apologise for the inconvenience caused to those affected. We assure you that we have taken and will continue to take steps to ensure that Stears maintains its role as the most reliable way to access insights, updates and results for Nigeria’s 2023 elections. 

In this note, I will clarify why it happened and what we are doing to fix it.

Root causes

Collecting data across a country as large, complex and opaque as Nigeria is challenging, especially during a tense period like general elections.

We had three data sources for the election; one sourced through the INEC Result Viewing Portal (IREV), another from a partnership with Independent Observers across all 774 Local Government Areas (LGAs) and the last through monitoring collation centres (state and national) across the country. IREV was designated as our primary data source.

After elections are concluded at polling units (176,000), the results are manually aggregated into wards (8,809) and subsequently further aggregated into local government areas (774). While most platforms only revealed state level data, our website and app presented data more granularly. Our data collection infrastructure was designed to collect data from the polling units, collate it up to ward and LGA levels, and then make it available to our users after verification.
 

However, as most users know, things went differently than planned with the IREV portal—where official results from all polling units across the country were supposed to be uploaded by the Electoral Commission. Our systems, pre-built to automatically collate data up to the state level as soon as they were retrieved from IREV, could not function due to the incomplete data on the portal. Hence, we could not compute the results automatically.

We were prepared for this possibility, and as soon as IREV data proved unreliable, we switched from our first data source (IREV) to our second data source (independent observers at all 774 LGAs) to continue providing results. Nevertheless, IREV was expected to be the quickest source of data. Switching data sources to independent observers lengthened the time-to-delivery of our data as we needed to wait for INEC collation to happen at the LGA level before receiving the results.

Question 1 - Why did our data appear to reduce on occasion?

As the elections progressed, we switched the data source powering our mobile and web platforms again from our second source (independent observers at all 774 LGAs) to our third and final source (the National Collation Centre in Abuja). We made this switch because the National Collation Centre at Abuja makes the final declaration of election results and all data sources before it is considered provisional or tentative.

In retrospect, such a significant data source switch should have been better communicated to users.

Due to the switch to collation centre data, some users noticed fewer votes than previously allocated to different candidates. The reason is simple: the declared results (state level) were different from the provisional results (LGA level) because we had more results available at the LGA level than at the state level at the time of the switch.

On Monday, we decided to present both “Declared” and “Provisional” data sources, ensuring that users could access all of our data.

Question 2 - Why were some publicly available results not immediately updated on Stears?

In an election, speed is essential. But accuracy, even more so. As a data company, we measure our value to users with speed, reliability, and accuracy.

In the trade-off between speed and accuracy, we are committed to publishing accurate and reliable data as quickly as possible. We believed, and still believe, that it was more important to be correct than fast. Therefore, data validation is not a process we take lightly. 

When we receive data from unofficial sources, we require an extra layer of checks to ensure that the data is accurate and that no clerical or non-clerical errors have affected the data input. The most reliable data source for elections is the PU/Ward/LGA result sheet. Our data entry staff and partners transcribe this data into data sheets via scripts or manual entry before quality assurance (QA) steps are taken.

Sometimes, mistakes are made during data entry, which must be identified, corrected and re-validated during QA. In trickier cases, we can only validate the data in QA with a readable version of the PU/Ward/LGA result sheet. Where the result sheets are unavailable, we rely on other external sources to verify the accuracy of the data. Before publishing the data, we need the result sheet or external verification to protect our users from misinformation.
 

^{Sample of a result sheet used for data verification}

On multiple occasions, this approach proved far-sighted, as additional verification of the results changed how hundreds of thousands of votes were allocated.

Question 3 - Why did I experience challenges when using the app or receiving notifications?

Our systems were overwhelmed with the traffic (both human and non-human) attempting to access information. For context, we saw a 5,000% increase in user visits and a nearly 40x increase in new mobile app installs during the elections. The growth across channels and platforms meant several of our services failed to perform optimally, putting pressure on an already stretched team.

While we prepared for regular web and mobile traffic to our election pages—which had no downtime—we were less prepared for the non-human traffic directed to other parts of the site, seemingly by other programmers or scripts, attempting to innocently or maliciously retrieve data at scale. While this traffic grew, our APIs which power mobile app logins, push notifications and content, were overwhelmed and failed to respond for long periods.

Our APIs react to growth in traffic by self-monitoring their resource usage and then automatically reacting to an increase in usage. However, upon investigation, we found that our monitoring tool, which needed additional resources to handle the traffic, did not do so due to an unexpected error in its configuration. We responded by manually increasing and decreasing resources to match the rise in traffic, which allowed us to reduce the impact on users eventually.

Prevention and follow-up

All our teams have been in constant conversation since we experienced the downtimes. We’ve organised multiple retrospectives to ensure that we understand what went wrong, what we should have done better, and identify actions to eliminate the chances of this happening again.

We have now taken steps to protect our APIs better and to improve the reliability of our monitoring. We will continue testing them regularly to identify other potential points of failure and remove any possible sources of delay or outage to our users.

We are also working on additional measures to protect the website and mobile app from the unwanted traffic that degraded the quality of service our users experienced last weekend. 

We have identified ways to present different data sources simultaneously to allow users to see what we see as quickly as we see it. We have now reallocated data collection resources to concurrently monitor and present various reliable data sources.

And finally, we have tightened our QA processes to minimise the risk or impact of human errors across our multiple workstreams.

Some days, our systems built to produce the most rigorous, accurate and reliable source of information on the continent work seamlessly. On other days, like we just experienced, we experience multifaceted challenges, which slow us down. We acknowledge that these occurrences, by necessity, ought to be rare, and we are committed to it.

This event has reminded us that in a low-trust environment like Nigeria, data companies like Stears are essential in ensuring trusted, reliable and accurate information is available. It is a responsibility that we take very seriously.

As we approach the Governorship elections, we will continue to complete the available data on the site and ensure that we are ready to provide a smooth, hitch-free experience on election day.