By Adam Conway
Your changes have been saved
Email is sent
Email has already been sent
Please verify your email address.
You’ve reached your account maximum for followed topics.
Manage Your List
Follow
Followed
Follow with Notifications
Follow
Unfollow
Link copied to clipboard
Qualcomm
Related
4 reasons SSHDs were never a viable replacement for HDDs or SSDs
6 reasons you should use Ubuntu for your own NAS
Can your ISP tell if you're using a VPN?
Just last week saw the advent of the new Qualcomm Snapdragon 8 Gen 1 at the company's tech summit in Hawaii. Qualcomm's latest chipset packs a whole host of major improvements across the board, making this one of the most exciting Qualcomm devices in a long time. While the company was hesitant to provide in-depth technical details in some aspects (including neglecting to mention an Adreno or Kryo version name), we were still able to run a gamut of popular benchmarks on the Snapdragon 8 Gen 1 reference device. These benchmarks help set the baseline of performance expectations for upcoming flagships in 2022, giving us one more thing to look forward to next year.
On the Snapdragon 8 Gen 1 reference device, we ran one holistic benchmark (AnTuTu), a CPU-centric benchmark (Geekbench), a GPU-centric benchmark (GFXBench), and MLPerf benchmarks. Each benchmark was run three times, and we took the average of the three results. Qualcomm had enabled a "UI Perf Mode" option by default that we disabled, as it effectively tries to force the benchmarking apps to run on Prime cores in order to eke out a slightly higher score in certain benchmarks. It is also worth noting that once we get our hands on a commercial device with the Qualcomm Snapdragon 8 Gen 1, we will be rerunning these benchmarks.
If you’re interested in reading up on all the specifications and features of the Qualcomm Snapdragon 8 Gen 1 mobile platform that the company has made available so far, then I recommend reading our explainer on the Qualcomm Snapdragon 8 Gen 1. For quick reference, I put together a chart comparing the key specifications of the Qualcomm Snapdragon 8 Gen 1 reference device compared to the other two references devices used in this benchmark comparison. You can find that chart below ahead of the benchmark results.
Qualcomm provided us with a set of expected benchmark scores based on their own testing. We used this for reference only, and a table is available at the bottom of this article containing the benchmark scores that Qualcomm expected the reference device to achieve.
About this article: Qualcomm sponsored my colleague, Rich Woods, to attend the Snapdragon Tech Summit in Kona, Hawaii. The company paid for his flight and hotel. However, Qualcomm did not have any input regarding the content of this article.
Qualcomm Snapdragon 8 Gen 1 Benchmark Results
Test Device Specifications
Qualcomm Snapdragon 8 Gen 1(Qualcomm Reference Device) | Qualcomm Snapdragon 888(Qualcomm Reference Device) | Qualcomm Snapdragon 865(Qualcomm Reference Device) | |
---|---|---|---|
CPU |
|
|
|
GPU | New Adreno GPU (unspecified) | Adreno 660 | Adreno 650 |
Display |
|
|
|
AI |
|
|
|
Memory | 8GB LPDDR5 @ 3200MHz, 16GB |
|
|
Storage | 512GB UFS 3.1 | 512GB UFS 3.0 | 128GB UFS 3.0 |
ISP |
|
|
|
Manufacturing Process | 4nm (likely Samsung) | 5nm (Samsung’s 5LPE) | 7nm (TSMC’s N7P) |
Software version | Android 12 | Android 11 | Android 10 |
Overview of the benchmarks. Click to expand.
Benchmarks Overview
- AnTuTu: This is a holistic benchmark. AnTuTu tests the CPU, GPU, and memory performance, while including both abstract tests and, as of late, relatable user experience simulations (for example, the subtest which involves scrolling through a ListView). The final score is weighted according to the designer’s considerations.
- GeekBench: A CPU-centric test that uses several computational workloads including encryption, compression (text and images), rendering, physics simulations, computer vision, ray tracing, speech recognition, and convolutional neural network inference on images. The score breakdown gives specific metrics. The final score is weighted according to the designer’s considerations, placing a large emphasis on integer performance (65%), then float performance (30%), and finally cryptography (5%).
- GFXBench: Aims to simulate video game graphics rendering using the latest APIs. Lots of onscreen effects and high-quality textures. Newer tests use Vulkan while legacy tests use OpenGL ES 3.1. The outputs are frames during test and frames per second (the other number divided by the test length, essentially), instead of a weighted score.
- Aztec Ruins: These tests are the most computationally heavy ones offered by GFXBench. Currently, top mobile chipsets cannot sustain 30 frames per second. Specifically, the test offers really high polygon count geometry, hardware tessellation, high-resolution textures, global illumination and plenty of shadow mapping, copious particle effects, as well as bloom and depth of field effects. Most of these techniques will stress the shader compute capabilities of the processor.
- Manhattan ES 3.0/3.1: This test remains relevant given that modern games have already arrived at its proposed graphical fidelity and implement the same kinds of techniques. It features complex geometry employing multiple render targets, reflections (cubic maps), mesh rendering, many deferred lighting sources, as well as bloom and depth of field in a post-processing pass.
- MLPerf Mobile:MLPerf Mobile is an open-source benchmark for testing mobile AI performance. It wascreated by MLCommons, a non-profit, open engineering consortium, to “deliver transparency and a level playing field for comparing ML systems, software, and solutions.” MLPerf Mobile’s first iteration provides an inference-performance benchmark for a handful of computer vision and natural language processing tasks. For more information, refer to the paper “MLPerf Mobile Inference Benchmark: Why Mobile AI Benchmarking Is Hard and What to Do About It.”
- Image classification:This test involves inferring a label to apply to an input image. Typical use cases include photo searches or text extraction. The reference model used is MobileNetEdgeTPU with 4M parameters, the dataset is ImageNet 2012 (224×224), and the quality target is 98% of FP32 (76.19% Top-1).
- Image segmentation:This test involves partitioning an input image into labeled objects. Typical use cases include self-driving or remote sensing. The reference model used is DeepLab v3+ with 2M parameters, the dataset is ADE20K (512×512), and the quality target is 93% of FP32 (0.244 mAP).
- Object detection:This test involves drawing bounding boxes around objects as well as providing a label for those objects. Typical use cases involve camera-input such as for hazard detection or traffic analysis while driving. The reference model is SSD-MobileNet v2 with 17M parameters, the dataset is COCO 2017 (300×300), and the quality target is 97% of FP32 (54.8% mIoU).
- Language processing:This test involves responding to questions colloquially. Typical use cases include online search engines. The reference model is MobileBERT with 25M parameters, the dataset is mini Squad (Stanford Question Answering Dataset) v1.1 dev, and the quality target is 93% of FP32 (93.98% F1).
read more
Benchmark Results
AnTuTu
Starting off with AnTuTu, we can see that the Qualcomm Snapdragon 8 Gen 1 reference device scored substantially higher than previous years, with a major year-on-year increase of roughly 240,000 points. This is a pretty major improvement of roughly 33%, well above Qualcomm's promises of a 20% speed improvement. This is only one test, but AnTuTu is a great tool for comparing raw computational ability between devices, even if not all of that translates into real-world usage.
Geekbench
In Geekbench 5.0, however, it's a different story entirely. The Snapdragon 8 Gen 1 demonstrates little to no gain over the Snapdragon 888 reference device from last, and in fact, even regresses in performance when it comes to multi-core. It's not as if we got a dud either -- Qualcomm gave us an informational page with benchmark scores to expect from the Snapdragon 8 Gen 1, and these fall in line with what should be expected. In fact, the single-core performance was higher than the upper-bounds that Qualcomm gave us, whereas the multi-core performance was 20 points below the lower-bounds of what to expect.
Having said that, I think it's clear that whatever the test is, it's not testing where Qualcomm has made improvements. Our other tests show major improvements over previous years' chipsets.
GFXBench
Qualcomm hasn’t disclosed much about the new Adreno GPU in the Snapdragon 8 Gen 1, so we have little to say about the GPU other than its gains in performance. We don't know the core count, we don't know the frequency, and we don't even have a version number. In GFXBench’s Manhattan test, which uses the OpenGL ES 3.0 API and renders a 1080p scene offscreen, the Snapdragon 8 Gen 1 had an average framerate of 221fps, about 31% and 75% higher than the framerates achieved by the Snapdragon 888 and 865 respectively. In GFXBench’s Aztec Ruins test, which uses the Vulkan graphics API and renders a 1080p scene offscreen, the Snapdragon 8 Gen 1 had an average framerate of 41fps. There's a caveat to this though; the previous results on reference devices were tested in 1080p, whereas the only Aztec Ruins test we had access to was in 1440p. Increasing to 1080p requires 43.75% more pixels to be rendered at a time, which is why the performance decreased in this test.
Only some of the very best Android games require a lot of GPU horsepower, but improved GPU performance is useful for more than just gaming. Having said that, gaming is definitely the biggest reason why people will care about these benchmark results, and the Snapdragon 8 Gen 1 seems to deliver with its 35% faster graphics rendering and 20% better power efficiency year on year. These results only demonstrate the peak GPU performance, however, so we’ll have to revisit GFXBench — once we get our hands on commercial hardware — in order to run the benchmark’s long-term performance tests.
MLPerf
Qualcomm has been particularly cagey about specifics when it comes to artificial intelligence improvements. We don't have any figures for TOPS (Trillion Operations Per Second), though we do have some information about other improvements. There's the third generation of the company's sensing hub for example, and it also demonstrated a number of other SoC-specific features in Hawaii.
It is difficult for us to demonstrate just how significant this leap in performance actually is, however. We’ve talked in-depth about the difficulties of AI benchmarking during our interviews withQualcomm’s Travis Lanier,Gary Brotman, and Ziad Asghar. The good news is that, since our discussions with Qualcomm execs, there have been significant advances in the field of AI benchmarks.
The most promising benchmarks currently available come in the form of MLPerf Mobile, which is an open-source mobile AI benchmark backed by multiple SoC vendors, ML framework providers, and model producers. Its initial batch of mobile inferencing results is public, so we used those results to compare the Snapdragon 8 Gen 1 to the Snapdragon 888 in the Xiaomi Mix 4, the Dimensity 1100 in the Vivo S9 5G, and the Exynos 2100 in the Samsung Galaxy S21 Plus. We didn't get latency results — only throughput figures — so we did not plot the full results as submitted by the vendorsfor verification by MLCommons.
In these select computer vision and natural language processing inferencing benchmarks, we can see that the Qualcomm Snapdragon 8 Gen 1 reference device achieved the highest scores in all four tests by far. The Demensity 1100 fared pretty poorly overall. Qualcomm’s Snapdragon 888 still comfortably beat the others in this test, but the Snapdragon 8 Gen 1 is on another level in all of these tests.
It will be interesting to see what applications and features developers and OEMs can create using the AI prowess of the Snapdragon 8 Gen 1. Computer vision will play an especially important role in the many AI-enhanced videography features we’ll likely see in 2022, while improved NLP performance can likewise affect video adjacent aspects like audio recording. Companies like Google working on Google Tensor will also push other SoC vendors in this department.
Conclusion
The table that Qualcomm provided us with expected benchmark scores is below, which you can see falls mostly in line with the results that we achieved above.
Expected benchmark scores for Snapdragon 8 Gen 1 reference design (from Qualcomm)
Benchmark | Version | Method | Expected Score Range | |
---|---|---|---|---|
System | Geekbench ST | v5.4.2 | Average of 3 iterations | ~1220 - 1233 |
System | Geekbench MT | v5.4.2 | Average of 3 iterations | ~3770 - 3810 |
System | AnTuTu | v9.2.1 | Average of 3 iterations | 1st run: ~1mAvg of 3 iterations: ~980K |
System | PCMark | v3.0.4061 | Average of 3 iterations | ~17k |
Browser (Chrome v95.0.4638.74 64-bit) | JetStream | v2.0 | Average of 3 iterations | ~135 - 140 |
Browser | SpeedoMeter | v2.0 | Average of 3 iterations | ~123 - 126 |
Browser | WebXPRT | v3.0 | Average of 3 iterations | ~194 - 197 |
AI | AITuTu | v2.0 | Average of 3 iterations | ~2,550,000 - 2,600,000 |
AI | AIMark | v3.0 | Average of 3 iterations | ~97K |
AI | MLPerf (on a separate QRD since it takes 30 mins to run) | v1.1 | Average of 3 iterations |
|
AI | ETH AIB | v4.0.4 | Average of 3 iterations | ~530 - 550k |
Graphics | GFXBench Manhattan 3.0 Offscreen (1080p) (FPS) | v5.0 | Average of 3 iterations | ~267 - 268 FPS |
Graphics | GFXBench T-Rex - Offscreen (1080p) (FPS) | v5.0 | Average of 3 iterations | ~450 - 452 FPS |
Graphics | GFXBench Manhattan 3.1 Offscreen (1080p) (FPS) | v5.0 | Average of 3 iterations | ~176 FPS |
Graphics | GFXBench Car Chase Offscreen (1080p) ES3.1 (FPS) | v5.0 | Average of 3 iterations | ~97 - 98 FPS |
Graphics | GFXBench Aztec Ruins Vulkan (High Tier) Offscreen (1440p) (FPS) | v5.0 | Average of 3 iterations | ~49 FPS |
Graphics | GFXBench Aztec Ruins OpenGL (High Tier) Offscreen (1440p) (FPS) | v5.0 | Average of 3 iterations | ~43 FPS |
read more
The Snapdragon 8 Gen 1 brings a ton of improvements over last year's chipset, especially when it comes to AI. While there is definitely some weirdness in the CPU-bound results from Geekbench, it's clear that there are across-the-board improvements. If you're upgrading from a device that's two years older (or older) then the improvements will likely be noticeable, though the massive gains in AI performance will likely go unnoticed by most. Companies seldom leverage the full potential of AI when it comes to Qualcomm's chipsets, and it's likely that will be the same again here.
It's also worth noting that with competition heating up, it might be worth waiting to see what Samsung and MediaTek do next. The Dimensity 9000 chipset has the potential to take the Snapdragon 8 Gen 1 head-on when it comes to performance, and there's not a lot that we actually know about the upcoming Exynos 2200 just yet. I'm personally looking forward to revisiting these benchmarks on a commercial device in the future, particularly in a more controlled setting.
- Qualcomm
- Best Posts
- Qualcomm Snapdragon 8 Gen 1
Your changes have been saved
Email is sent
Email has already been sent
Please verify your email address.
You’ve reached your account maximum for followed topics.
Manage Your List
Follow
Followed
Follow with Notifications
Follow
Unfollow
Readers like you help support XDA. When you make a purchase using links on our site, we may earn an affiliate commission. Read More.