DULLES, Va.--(BUSINESS WIRE)--Global Wireless Solutions, Inc. (GWS) and Hiya today shared findings from a study that assessed how well wireless carrier spam detection solution providers (Hiya, First Orion and TNS) detect unwanted robocalls. This study introduces a first-of-its-kind methodology designed by Hiya to set a new industry standard, focusing specifically on spam “identification rate” and spam “error rate,” and assigning an overall “detection score” based on the combination of these two metrics.
The results of this study revealed the following:
-
Identification Rate (portion of all confirmed spam phone
numbers in the dataset tested that the provider correctly classified
as spam):
- For this study, Hiya’s identification rate was 87%, First Orion’s was 55% and TNS’ was 34%.
-
Error Rate (portion of all calls that a provider classified as
spam, that were not actually spam, i.e. false positives):
- For this study, Hiya’s error rate was 0.25%, First Orion’s was 1.26% and TNS’ was 1.48%.
-
Detection Score (a calculation that takes the Identification
Rate and the Error Rate and applies a penalty for incorrectly
classifying a number as spam):
- The overall Detection Score for Hiya was 85 versus First Orion’s at 48 and TNS’ at 26.
New Industry Standard for Evaluating Spam Detection Capabilities
In order to provide the most scientific way of testing each spam detection provider’s ability to accurately detect spam calls, Hiya developed a unique and first-of-its-kind approach to the testing methodology. Instead of testing static “lists” of known spam numbers, they examined actual phone traffic over the course of 24 hours and leveraged the resulting set of numbers to test and determine how the providers classified each number. GWS worked with Hiya by manually verifying the number classifications that did not correlate across all three of the solution providers. A detailed explanation of the methodology and why Hiya is suggesting it as the new industry standard for testing can be found here.
The Rise in Robocalls and the Importance of Accurately Detecting Spam
The rise in robocalls continues to be a major issue affecting American consumers and regulators. FCC Chairman Ajit Pai recently threatened regulation if US carriers don’t work faster to adopt a robust call authentication system (STIR/SHAKEN) to combat illegal caller ID spoofing. A bipartisan group of senators also recently re-introduced the TRACED Act, a bill that would dramatically increase the fines faced by robocallers. The issue of accuracy and correctly identifying a call – whether wanted or unwanted – is important both to protect consumers from annoying spam and potentially harmful scams, but also so they don’t miss important calls from numbers they might not otherwise recognize (schools, doctor’s office, emergency services, etc.).
Methodology Overview:
The study was conducted in December 2018 by GWS and Hiya to compare the spam filtering capabilities of Hiya, First Orion and TNS. This study was based on an extensive list of phone numbers gathered by Hiya from a sampling of neutral network traffic instead of using a static list of known spam numbers. The preliminary tests performed by Hiya evaluated over 5,000 phone numbers to determine the caller ID classification assigned by each spam detection provider, ultimately categorizing the numbers as some form of spam or not. This “call down” process involved synchronized dialing through an automated system to multiple mobile phones with spam detection software installed and documenting the caller ID label. From the initial numbers tested, a subset of 751 numbers was identified where the three providers’ caller ID label disagreed on whether the call was spam or not. Those numbers were provided to GWS to apply manual verification via online research and verification calls if needed to determine the correct classification (spam or not spam).
About Global Wireless Solutions, Inc.:
Founded in 1996 and headquartered in Dulles, Virginia, Global Wireless Solutions has set the industry standard for network benchmarking, analysis and testing. Its proprietary OneScore network ranking combines high-level network testing data with real-world network usage scenarios to make the results of testing relatable and easy to understand. Working with some of the world’s largest wireless network providers, GWS offers high-quality network data and engineering analysis through a suite of benchmarking products, services and OneMeasure™ diagnostic apps that include drive, venue and in-building testing. At last count, GWS has driven 12 million data collection miles for its customers. For more information, visit gwsolutions.com and follow us on Twitter at @gwsolutionsinc.