Wednesday, August 12, 2026
Laptops

What are CUDA Cores?

Miguel Salas

CUDA core count is one of the first numbers you’ll see when comparing NVIDIA graphics cards.

One GPU may have 2,500 CUDA cores while another has 8,000 or more. That makes it tempting to buy whichever GPU has the largest number.

That’s not how it works.

More CUDA cores can mean more processing power, but CUDA core count is only one part of a GPU. The GPU architecture, VRAM, memory bandwidth, clock speed, wattage and laptop cooling system can be just as important.

This guide has been divided into two parts:

  • The Beginner Section explains what CUDA cores mean when choosing a laptop or GPU for gaming, data science, machine learning and professional software.
  • The Technical Section explains what CUDA cores physically are, where they are located and how CUDA threads are actually processed inside the GPU.

[toc]

Part 1: CUDA Cores for Beginners

What Is a CUDA Core?

A CUDA core is a small calculation unit inside an NVIDIA graphics processor.

Its job is to perform mathematical operations.

An NVIDIA GPU contains thousands of these units because games, image-processing programs, data-science tools and other GPU applications need to repeat similar calculations across huge amounts of data.

Simple explanation: CUDA cores are the general-purpose mathematical workers inside an NVIDIA GPU.

They are used to process things such as:

  • Game lighting and graphical effects
  • Pixels, textures and 3D objects
  • Dataframe operations
  • Image and video filters
  • Scientific simulations
  • 3D rendering
  • Machine-learning calculations
  • Large groups of numbers that can be processed in parallel

However, a CUDA core is not a complete processor by itself.

It works as part of a larger group of GPU hardware called a Streaming Multiprocessor.

Where Are CUDA Cores Located?

CUDA cores are located inside sections of the NVIDIA GPU called Streaming Multiprocessors, usually shortened to SMs.

A simplified layout looks like this:

NVIDIA GPU → Streaming Multiprocessors → CUDA Cores

Each Streaming Multiprocessor contains a group of CUDA cores plus the hardware needed to organize their work.

This includes:

  • Instruction schedulers
  • Registers
  • Cache
  • Shared memory
  • Load and store units
  • Tensor Cores
  • Other calculation units

The CUDA cores do the basic arithmetic, but the complete Streaming Multiprocessor decides which calculations should be performed and supplies the necessary data.

Image source: NVIDIA Streaming Multiprocessor and GPU architecture diagrams

CUDA Core vs CPU Core

A CUDA core is not the same thing as an Intel Core, AMD Ryzen core or Apple CPU core.

A CPU core is a powerful and relatively independent processor. It can handle complicated instructions, unpredictable software, operating-system tasks and heavily branched programs.

A CUDA core is much simpler. It performs mathematical instructions as part of a large group controlled by the Streaming Multiprocessor.

Feature CPU Core CUDA Core
Main job Finish complicated software instructions quickly Perform repeated mathematical operations
Independent processor Yes No
Typical quantity A few to several dozen Thousands across the GPU
Best type of work Serial and unpredictable tasks Large amounts of parallel math

An eight-core CPU is not weaker than a GPU simply because the GPU has 5,000 CUDA cores.

The two numbers describe completely different types of hardware.

What Does CUDA Mean?

CUDA stands for Compute Unified Device Architecture.

It is NVIDIA’s platform for allowing software to use an NVIDIA GPU for general calculations.

CUDA cores are the physical calculation units.

CUDA is the software platform that allows programs to use the GPU for things other than regular graphics.

CUDA and CUDA cores are not the same thing. CUDA is the programming platform. CUDA cores are part of the physical GPU.

A program does not necessarily have to display graphics to use an NVIDIA GPU.

CUDA can be used for:

  • Machine learning
  • Scientific computing
  • Data analysis
  • 3D rendering
  • Medical imaging
  • Engineering simulations
  • Financial calculations
  • Image and video processing

Do Games Use CUDA Cores?

Yes.

CUDA-core hardware performs much of the ordinary graphics math used by NVIDIA GPUs.

In games, these calculation units are used for:

  • Processing 3D objects
  • Calculating lighting
  • Applying shadows
  • Processing pixels
  • Running graphical shaders
  • Calculating materials and reflections
  • Processing particles
  • Applying post-processing effects

A game does not have to be specifically advertised as a CUDA application.

Graphics APIs such as DirectX and Vulkan can use the same general GPU calculation hardware.

Do more CUDA cores mean more FPS?

More CUDA cores usually help when comparing GPUs from the same architecture.

For example, if two GPUs have:

  • The same architecture
  • Similar clock speeds
  • Similar memory
  • Similar wattage

The GPU with more CUDA cores will normally have more graphics-processing power.

However, game performance also depends on:

  • GPU architecture
  • Clock speed
  • VRAM
  • Memory bandwidth
  • Cache
  • RT Cores
  • Tensor Cores
  • GPU wattage
  • Cooling
  • The laptop’s CPU
  • The game and graphics settings

This is why you should use actual gaming benchmarks instead of choosing a GPU from CUDA core count alone.

A game may also be limited by the CPU, VRAM or another part of the graphics pipeline. The GPU bottleneck guide explains how these limitations affect gaming performance.

CUDA Cores vs Tensor Cores vs RT Cores

Modern NVIDIA RTX GPUs contain more than one type of calculation hardware.

GPU Hardware Main Job Common Uses
CUDA Cores General mathematical calculations Shaders, data processing, simulations and regular GPU calculations
Tensor Cores Matrix calculations Deep learning, AI processing and neural graphics
RT Cores Ray-tracing calculations Ray-traced lighting, reflections and shadows

These units work together.

In a ray-traced game, RT Cores may calculate where rays hit objects. CUDA cores still process shaders, materials and lighting. Tensor Cores may then help with AI reconstruction or denoising.

The DLSS and FSR guide explains how AI reconstruction is used to increase gaming performance.

What Do CUDA Cores Do for Data Science?

Data science often involves applying the same operation to a large number of values.

For example, you may need to:

  • Filter millions of rows
  • Transform dataframe columns
  • Sort large datasets
  • Calculate distances between data points
  • Perform matrix calculations
  • Group or join large tables
  • Train a machine-learning model
  • Run clustering or dimensionality-reduction algorithms

A GPU can divide these operations between thousands of threads.

This can make suitable workloads much faster than processing the data with a small number of CPU cores.

However, owning an NVIDIA GPU does not automatically make every Python program faster.

Your software must actually support GPU acceleration

Regular pandas, NumPy or scikit-learn code may continue running on the CPU unless you use a GPU-enabled library or feature.

Common GPU-accelerated tools include:

  • CuPy
  • cuDF
  • cuML
  • cuGraph
  • PyTorch
  • TensorFlow
  • Numba CUDA
  • CUDA-enabled XGBoost

Some of these tools have interfaces similar to familiar CPU libraries, but the program still needs to be configured to use the GPU.

The first thing a data-science student should check is not CUDA core count.

The first thing to check is whether the software, library or project actually supports CUDA.

When GPU acceleration makes a big difference

GPU acceleration becomes more useful when:

  • The dataset is large
  • The calculation can be divided into many similar operations
  • The data can remain in GPU memory
  • The software has a well-optimized GPU implementation
  • The calculation takes long enough to justify sending it to the GPU

Researchers developing a GPU version of UMAP reported speedups reaching approximately 100 times in the workloads they tested.

Research on GPU-accelerated single-cell analysis has also reported speedups of up to several hundred times for certain parts of large analysis pipelines.

These are examples, not promises.

Your result depends on the dataset, software, GPU, CPU baseline and how much data must be transferred between system RAM and VRAM.

When a CPU may still be faster

A CPU may be faster when:

  • The dataset is small
  • The operation is mostly serial
  • The code contains many unpredictable decisions
  • The GPU version of the library is poorly optimized
  • Data must constantly move between the CPU and GPU
  • The operation finishes before GPU acceleration has time to help

Using a GPU has overhead.

The data may need to be copied to the GPU, the calculation must be launched and the result may need to be transferred back.

For a tiny job, that process can take longer than simply letting the CPU handle it.

What Do CUDA Cores Do in Machine Learning?

CUDA cores are important for machine learning, but Tensor Cores are also a major part of modern AI performance.

Tensor Cores specialize in the matrix calculations used throughout neural networks.

CUDA cores handle many of the other operations around those matrix calculations, including:

  • Data preparation
  • Tensor indexing
  • Activation functions
  • Element-by-element math
  • Loss calculations
  • Normalization
  • Custom GPU kernels
  • Operations that cannot use Tensor Cores

A GPU with more CUDA cores may be faster, but CUDA core count is not the best single specification for choosing a deep-learning laptop.

VRAM capacity, Tensor Core generation and memory bandwidth can be more important.

What Matters Most When Choosing a Data-Science Laptop?

If you need GPU acceleration, compare the following specifications in this order.

1. Software compatibility

Check whether your software requires NVIDIA CUDA.

If it does, you need an NVIDIA GPU.

AMD, Intel and Apple GPUs do not contain CUDA cores and cannot run CUDA software natively.

This does not mean those GPUs are useless. They have their own computing platforms and can run software written for them.

However, if your university course, employer, research project or library explicitly requires CUDA, the safest choice is an NVIDIA GPU.

2. VRAM capacity

VRAM is the dedicated memory attached to the GPU.

It stores:

  • Datasets
  • Model parameters
  • Training batches
  • Activations
  • Gradients
  • Temporary calculations

If the required data does not fit in VRAM, the program may:

  • Fail with an out-of-memory error
  • Require a smaller batch size
  • Use slower CPU offloading
  • Split the job into smaller pieces

For local data-science and machine-learning work, buying slightly more VRAM is often more useful than buying a GPU with more CUDA cores but less memory.

For machine learning, the fastest GPU is not always the best choice. A slightly slower GPU with more VRAM may be able to run projects that do not fit on the faster GPU.

3. The actual GPU model and generation

Compare the complete GPU model rather than CUDA core count alone.

A newer GPU can outperform an older one despite having fewer CUDA cores because it may have:

  • Higher clock speeds
  • Better cache
  • Improved scheduling
  • Newer Tensor Cores
  • Higher efficiency
  • Better software support

The difference between architecture and core count can be seen when comparing the RTX 3060 and RTX 4050 Laptop GPUs.

4. GPU wattage or TGP

Laptop GPUs can operate at different power limits.

Two laptops may advertise the same NVIDIA GPU and the same CUDA core count but deliver different performance.

A higher power limit normally allows the GPU to sustain higher clock speeds.

A lower-power version may run slower even though it contains the same number of CUDA cores.

Before buying a laptop, check the laptop GPU TGP guide.

5. Cooling

A laptop needs to remove the heat produced by its CPU and GPU.

If the cooling system cannot keep up, the GPU may reduce its clock speed.

A thin laptop with a powerful GPU can perform well during a short test and then slow down during a long training, rendering or simulation job.

A larger laptop with better fans, heat pipes and ventilation may maintain better performance over time.

The laptop cooling-system guide explains the components responsible for sustained performance.

6. System RAM

Your data normally starts in system RAM before being transferred to VRAM.

System RAM is also used for:

  • Python
  • Jupyter notebooks
  • Data preparation
  • CPU-based libraries
  • Multiple applications
  • Datasets that do not fit entirely on the GPU

A powerful NVIDIA GPU does not compensate for a laptop that constantly runs out of system memory.

7. CPU performance

The GPU does not replace the CPU.

The CPU still handles:

  • Running the operating system
  • Starting GPU jobs
  • Preparing data
  • Reading files
  • CPU-only libraries
  • Code that cannot run in parallel

A weak CPU can delay data preparation or leave the GPU waiting for work.

8. SSD capacity

Datasets, environments, model checkpoints and virtual machines can use a surprising amount of storage.

A fast SSD also helps load large datasets and move files between projects.

The SSD does not directly increase CUDA core speed, but a small or slow drive can still make the complete workflow frustrating.

The beginner guide to laptop specifications explains how the CPU, RAM, storage and dedicated GPU work together.

Practical GPU and VRAM Targets

There is no exact CUDA core requirement that applies to everyone.

The following are general starting points rather than strict rules:

Workload Practical Starting Point Main Limitation
Learning CUDA or basic GPU programming Any reasonably recent CUDA-compatible NVIDIA GPU Software compatibility matters more than maximum performance
General data science A modern mid-range NVIDIA RTX GPU with at least 6GB to 8GB of VRAM Many common tools may still run on the CPU
Moderate machine learning 8GB to 12GB of VRAM, with more preferred Model and batch size
Computer vision or larger local models 12GB to 16GB or more VRAM where the budget allows Images, activations and model parameters consume large amounts of memory
Gaming Choose using game benchmarks at your target resolution Core count alone cannot predict FPS

These numbers depend heavily on the project.

A small neural network may run on a low-end GPU. A large model may not fit on the most powerful laptop GPU available.

If your largest projects will only be run occasionally, buying a cheaper laptop and renting cloud GPUs may make more sense than carrying an expensive and heavy workstation every day.

For more workload-specific recommendations, see the LaptopStudy data-science laptop guide.

Beginner Laptop-Buying Checklist

Before buying a laptop for CUDA, data science or machine learning, answer these questions:

  1. Does my software specifically support or require NVIDIA CUDA?
  2. How much VRAM do my datasets or models need?
  3. Is the GPU a recent architecture?
  4. What is the laptop GPU’s wattage or TGP?
  5. Can the cooling system sustain long GPU workloads?
  6. Does the laptop have enough system RAM?
  7. Can the RAM and storage be upgraded?
  8. Do benchmarks exist for my actual program?
  9. Would a desktop or cloud GPU provide better value?

Beginner conclusion: do not buy a laptop because it has a big CUDA core number. First check software compatibility and VRAM. Then compare the GPU model, wattage, cooling and real benchmarks.

Part 2: Technical Guide to CUDA Cores

CUDA Cores and the CUDA Platform

The term CUDA is used in two related but different ways.

CUDA is NVIDIA’s parallel-computing platform, programming model, compiler toolchain and collection of software libraries.

CUDA cores are arithmetic execution units inside NVIDIA GPUs.

The CUDA platform allows developers to write programs in which the CPU controls the main application while suitable calculations are sent to the GPU.

The CPU side is normally called the host.

The GPU side is called the device.

A typical CUDA application performs the following steps:

  1. The CPU starts the program.
  2. Memory is allocated on the CPU and GPU.
  3. Input data is copied to the GPU when required.
  4. The CPU launches a GPU function.
  5. The GPU processes the data using many threads.
  6. The result is copied back or used by another GPU operation.

A function launched on the GPU is called a kernel.

Image source: NVIDIA CUDA kernel, grid, block and thread diagrams

What Does a CUDA Core Physically Do?

A CUDA core is best understood as an arithmetic execution lane.

It processes common mathematical instructions issued by the Streaming Multiprocessor.

These instructions can include:

  • 32-bit floating-point addition
  • 32-bit floating-point multiplication
  • Fused multiply-add
  • Supported integer operations
  • Logical operations
  • Format conversions

The exact capabilities vary between NVIDIA architectures.

The advertised CUDA core count has traditionally been associated primarily with the GPU’s FP32 arithmetic lanes.

Some architectures have separate FP32 and integer pipelines. Other architectures allow some execution lanes to process either FP32 or supported integer instructions.

A CUDA core should therefore be treated as an architecture-specific execution resource rather than a permanently identical component across every NVIDIA generation.

Fused multiply-add

A common GPU instruction is fused multiply-add:

D = A × B + C

The GPU multiplies two values and adds a third value as one instruction.

This is usually counted as two floating-point operations:

  • One multiplication
  • One addition

This convention is used when calculating theoretical TFLOPS.

The Streaming Multiprocessor

A CUDA core does not independently retrieve, decode and schedule an entire software thread.

It depends on the surrounding Streaming Multiprocessor.

An SM may contain:

  • FP32 execution lanes
  • Integer execution lanes
  • Warp schedulers
  • Instruction dispatch units
  • Tensor Cores
  • Load and store units
  • Special Function Units
  • Texture units
  • Registers
  • L1 cache
  • Shared memory

The SM performs several important jobs:

  • Accepting thread blocks
  • Dividing threads into warps
  • Tracking the state of resident threads
  • Selecting ready warps
  • Issuing instructions
  • Managing registers and shared memory
  • Sending memory requests
  • Dispatching instructions to the correct execution units

A complete GPU contains many SMs.

Increasing the active SM count usually increases the number of CUDA cores, Tensor Cores, texture units and other resources available across the chip.

CUDA’s Thread Hierarchy

CUDA organizes software work into several levels.

Level Purpose
Kernel The function launched for execution on the GPU
Grid All thread blocks created by one kernel launch
Thread Block A group of threads that can cooperate and use shared memory
Warp A hardware execution group containing 32 threads
Thread One logical instance of the kernel

When the CPU launches a kernel, it specifies the number of thread blocks and threads per block.

The GPU distributes those blocks among its available SMs.

A block is assigned to one SM and normally remains there until it finishes.

The threads inside the block can:

  • Use the same shared-memory region
  • Synchronize at defined points
  • Exchange intermediate results

Threads in separate blocks generally cannot use ordinary block-level synchronization with one another.

Warps and SIMT Execution

Threads inside a block are organized into groups of 32 called warps.

The warp is the main scheduling group used by the traditional CUDA execution model.

A warp scheduler selects a ready warp and issues its next instruction.

The active threads in that warp execute the same instruction on their own data.

This is called Single Instruction, Multiple Threads or SIMT.

Each thread still has its own:

  • Thread ID
  • Register values
  • Memory addresses
  • Logical program state

For example, all 32 threads in a warp may execute an addition instruction, but each thread adds a different pair of numbers.

Does One CUDA Thread Run on One CUDA Core?

Not in a permanent one-thread-per-core relationship.

A GPU may launch millions of logical threads while physically containing only several thousand CUDA cores.

The threads are processed in blocks, warps and waves.

The SM can keep many warps resident at once. Their instructions are dispatched to the available execution pipelines over time.

A logical thread may execute one instruction, wait while another warp runs and continue later.

This allows the GPU to process far more threads than it has physical arithmetic lanes.

Vector-addition example

Consider two arrays containing one million numbers:

C[i] = A[i] + B[i]

A simplified CUDA kernel can assign one array element to each thread:

__global__ void addVectors(float *a, float *b, float *c, int size)
{
    int i = blockIdx.x * blockDim.x + threadIdx.x;

    if (i < size)
    {
        c[i] = a[i] + b[i];
    }
}

The program may launch approximately one million logical threads.

Those threads are divided into blocks and then into 32-thread warps.

The available SMs process them in waves until the complete grid has finished.

Latency Hiding

Accessing VRAM takes much longer than performing a simple arithmetic operation.

A CPU tries to reduce this delay using large caches, branch prediction and out-of-order execution.

A GPU relies heavily on having many warps ready.

Suppose one warp requests data from VRAM and cannot continue.

The SM scheduler can issue an instruction from another ready warp instead of leaving the execution units completely idle.

This is called latency hiding.

The memory delay still exists. The GPU hides part of it by working on something else.

Efficient GPU kernels provide enough active warps to keep the execution pipelines busy while other warps wait.

Occupancy

Occupancy describes how many warps are resident on an SM relative to the maximum supported by that architecture.

Higher occupancy gives the scheduler more warps to choose from.

Occupancy may be limited by:

  • Registers used by each thread
  • Shared memory used by each block
  • Threads per block
  • The maximum resident block count
  • The maximum resident warp count

A kernel that uses too many registers per thread may allow fewer threads to remain resident.

A block that uses a large amount of shared memory may prevent additional blocks from fitting on the SM.

Maximum occupancy does not automatically mean maximum performance.

A well-designed compute-heavy kernel may perform well with lower occupancy. Occupancy only needs to be high enough to hide the workload’s important delays.

Warp Divergence

Warps are most efficient when their threads follow the same instruction path.

Consider the following condition:

if (value > 0)
{
    calculationA();
}
else
{
    calculationB();
}

Suppose half the warp chooses calculation A and the other half chooses calculation B.

The GPU may need to process both paths while disabling the threads that do not belong to the current path.

While path A runs, the path-B threads are inactive.

While path B runs, the path-A threads are inactive.

This is called warp divergence.

The program still produces the correct result, but the execution lanes are used less efficiently.

CUDA core count has not changed, but effective performance has fallen.

GPU Memory Hierarchy

CUDA cores need a constant supply of data.

NVIDIA GPUs use several levels of storage with different capacities and speeds.

Registers

Registers are the fastest storage available to GPU threads.

They hold values currently used by calculations.

Each SM contains a large register file shared among its resident threads.

Using more registers can reduce unnecessary memory access, but excessive register use can lower occupancy.

Shared memory

Shared memory is fast, on-chip storage shared by the threads in one block.

It can be used to:

  • Reuse frequently accessed data
  • Exchange intermediate results
  • Reduce repeated VRAM requests
  • Reorganize data into a more efficient access pattern

L1 and L2 cache

Caches retain recently accessed data.

A cache hit can reduce demand on the GPU’s external memory.

Cache capacity and behavior vary between NVIDIA architectures, which is another reason identical CUDA core counts do not guarantee identical performance.

Global memory or VRAM

VRAM is the GPU’s main memory.

It provides much more capacity than registers or shared memory but has much greater access latency.

Large datasets, textures, model parameters and output arrays are normally stored in VRAM.

Image source: NVIDIA CUDA memory hierarchy diagrams

Memory Coalescing

A GPU memory request is more efficient when neighboring threads access neighboring memory addresses.

Suppose the 32 threads in a warp read 32 consecutive values.

The GPU can often combine those requests into a small number of organized memory transactions.

If the same threads access addresses scattered across VRAM, more transactions may be required.

The arithmetic has not changed, but the scattered version consumes more bandwidth and takes longer.

Organizing global-memory access is called memory coalescing.

Coalesced access is one of the most important CUDA performance optimizations.

Compute-Bound vs Memory-Bound Workloads

A program is not necessarily limited by CUDA core count.

GPU workloads can be roughly divided into two groups.

Workload Type Main Limitation
Compute-bound The execution units cannot complete the mathematical operations any faster
Memory-bound The execution units spend time waiting for data

Consider a kernel that loads one number, adds one and writes the result back.

The calculation is extremely easy.

The program may spend more time moving the number than performing the addition. This is a memory-bound workload.

Adding more CUDA cores may provide little improvement if the memory system is already saturated.

Now consider a kernel that loads one number and performs hundreds of calculations with it.

This kernel has greater arithmetic intensity and is more likely to benefit from additional compute throughput.

The Roofline performance model describes this relationship between arithmetic throughput and memory traffic.

CUDA Cores and Theoretical TFLOPS

A simplified estimate of peak FP32 performance is:

FP32 Throughput = CUDA Cores × Clock Speed × Operations per Clock

If one execution lane completes one fused multiply-add per clock, the operation is normally counted as two floating-point operations.

Consider a hypothetical GPU with:

  • 5,000 CUDA cores
  • A 2GHz clock speed
  • Two counted operations per clock

The simplified calculation is:

5,000 × 2,000,000,000 × 2 = 20,000,000,000,000 operations per second

This equals approximately 20 FP32 TFLOPS.

This number is a theoretical peak.

It assumes that:

  • The relevant execution lanes remain active
  • The program supplies suitable FP32 instructions
  • The GPU maintains the stated clock speed
  • Memory and other hardware do not become bottlenecks

Real programs also perform:

  • Memory operations
  • Integer calculations
  • Branches
  • Synchronization
  • Address calculations
  • Control operations

Real application performance will therefore be lower than the theoretical peak.

CUDA Cores, Tensor Cores and RT Cores

The GPU sends different instructions to different execution units.

If the program requires a general FP32 calculation, CUDA-core arithmetic lanes may process it.

If the program requires supported matrix multiply-and-accumulate operations, Tensor Cores may process them.

If a game or renderer requires ray traversal or geometry intersection tests, RT Cores may accelerate those calculations.

Other instructions may be sent to:

  • Integer pipelines
  • Load and store units
  • Texture units
  • Special Function Units

A real GPU workload normally uses a combination of execution units.

This is why Tensor Core, RT Core and CUDA core counts cannot be added together to create one meaningful total.

Why CUDA Core Counts Change Between Architectures

NVIDIA has changed the design of its Streaming Multiprocessors between GPU generations.

For example, Turing SMs contained separate FP32 and INT32 pipelines.

Consumer Ampere GPUs added FP32 capability to a second datapath. This significantly increased their advertised FP32 CUDA core count and theoretical FP32 throughput.

That change did not make every program exactly twice as fast because real workloads use a mixture of floating-point, integer, memory and control instructions.

Newer architectures have continued changing:

  • How FP32 and integer instructions share pipelines
  • Cache capacity
  • Instruction scheduling
  • Tensor Core formats
  • RT Core capabilities
  • Power efficiency

Therefore, CUDA core counts are most useful when comparing GPUs from the same architecture.

When comparing different generations, use real benchmarks from the software or game you plan to run.

How CUDA Cores Are Used in Gaming

In conventional rasterized rendering, CUDA-core arithmetic hardware performs much of the programmable shader work.

This may include:

  • Vertex transformations
  • Pixel and fragment calculations
  • Lighting equations
  • Material calculations
  • Shadow processing
  • Particle systems
  • Ambient occlusion
  • Compute shaders
  • Post-processing

A shader may be executed for every vertex, pixel, particle or sample.

That produces enormous amounts of similar parallel work.

Ray-traced rendering adds additional hardware.

RT Cores accelerate ray and geometry operations. CUDA cores execute shaders and material calculations around those rays. Tensor Cores may support reconstruction, denoising or other neural rendering features.

How CUDA Cores Are Used in Data Science

Data-science libraries use CUDA kernels to divide large operations into many smaller pieces.

For a dataframe column, different threads may process different rows.

For a distance calculation, different threads may process different points or sections of a matrix.

For clustering or graph analysis, different blocks may process sections of the graph or groups of neighboring points.

The complete workflow may include:

  1. Loading the dataset into system RAM.
  2. Cleaning or preprocessing some data on the CPU.
  3. Transferring the required data to GPU memory.
  4. Launching one or more CUDA kernels.
  5. Keeping intermediate results in VRAM.
  6. Copying the final result back to the CPU when required.

Keeping data on the GPU between operations is important.

If every step repeatedly moves data between system RAM and VRAM, transfer overhead can eliminate much of the performance advantage.

Well-designed GPU data-science libraries attempt to keep compatible data structures in GPU memory throughout the pipeline.

Why Some Algorithms Are Difficult to Move to a GPU

Moving an existing CPU algorithm to CUDA is not always as simple as creating more threads.

The algorithm may require:

  • New data structures
  • Different memory layouts
  • Reduced synchronization
  • More regular control flow
  • More parallel versions of serial steps
  • Techniques for keeping data in VRAM

The GPU implementation of UMAP is a good example.

The researchers did not merely copy the original CPU code to the GPU. They redesigned important graph and optimization operations to suit GPU execution.

This is why software quality can matter as much as the GPU specification.

Double-Precision Performance

Not every scientific workload primarily uses 32-bit floating-point calculations.

Some engineering and scientific programs require 64-bit floating-point precision, known as FP64.

Consumer GeForce GPUs are usually designed around gaming and FP32 performance. Their FP64 throughput may be much lower relative to their FP32 throughput.

Professional and data-center GPUs may include much stronger FP64 hardware.

A GeForce GPU with more CUDA cores can therefore lose badly to a professional GPU in an FP64-heavy scientific workload.

If your software requires double precision, check the GPU’s actual FP64 specifications and benchmarks instead of relying on the advertised CUDA core count.

Laptop GPU Power and Cooling

A laptop GPU may contain the same number of CUDA cores as another laptop GPU with the same model name but run at a different wattage.

Higher wattage usually allows the GPU to maintain higher clock speeds during demanding work.

However, the laptop must also remove the extra heat.

Long GPU workloads such as:

  • Model training
  • Rendering
  • Scientific simulations
  • Long gaming sessions

can expose cooling limitations that are not obvious during a short benchmark.

When comparing laptops with the same GPU, check:

  • Maximum and sustained GPU wattage
  • Sustained clock speed
  • Fan configuration
  • Heat pipes or vapor chamber
  • Ventilation
  • CPU and GPU power sharing
  • Long-duration benchmarks

After buying a laptop, the GPU power-limit guide shows how to check its configured wattage.

When More CUDA Cores Help

More CUDA cores are most useful when:

  • The comparison uses the same or a very similar architecture
  • The workload is compute-bound
  • The program exposes enough parallel work
  • The memory system can keep the cores supplied
  • The GPU has enough VRAM
  • The cooling system maintains high clock speeds
  • The software is properly optimized

When More CUDA Cores Do Not Help Much

Extra CUDA cores may provide little improvement when:

  • The workload is memory-bound
  • The dataset is too small
  • The software runs primarily on the CPU
  • The algorithm contains large serial sections
  • Threads follow heavily divergent branches
  • The GPU repeatedly transfers data to and from the CPU
  • The laptop uses a low GPU power limit
  • The GPU is thermally throttling
  • VRAM capacity is insufficient
  • Tensor Core or FP64 performance is the real requirement

Frequently Asked Questions

Do AMD GPUs have CUDA cores?

No.

CUDA is NVIDIA technology.

AMD GPUs contain their own shader-processing hardware and use different programming platforms.

An AMD stream-processor count cannot be directly compared with an NVIDIA CUDA core count.

Do Intel GPUs have CUDA cores?

No.

Intel GPUs use Intel’s own execution hardware and software platforms.

Do Apple Silicon GPUs support CUDA?

No.

Apple Silicon GPUs use Apple’s Metal platform rather than CUDA.

Can a game use CUDA cores without using CUDA?

Yes.

Graphics shaders submitted through DirectX, Vulkan or OpenGL use the same general arithmetic hardware advertised as CUDA cores.

Do CUDA cores store data?

No.

Data is stored in registers, caches, shared memory and VRAM.

CUDA cores perform calculations using the values supplied by the SM.

Are CUDA cores only used for FP32 calculations?

The advertised CUDA core count is mainly associated with FP32 arithmetic capacity.

However, exact instruction support and the arrangement of integer and floating-point pipelines vary by architecture.

Can CUDA cores compensate for low VRAM?

No.

More arithmetic power does not create more memory.

If a model or dataset does not fit in VRAM, the software must use a smaller workload, divide the job or move data to slower system RAM.

Can CUDA cores compensate for low laptop wattage?

Not completely.

A GPU may contain many CUDA cores but run them at lower sustained clock speeds when its power limit is low.

Research Papers and Technical Sources

  1. NVIDIA CUDA Programming Guide
  2. NVIDIA CUDA Best Practices Guide
  3. NVIDIA CUDA Toolkit
  4. NVIDIA RTX Blackwell GPU Architecture
  5. Scalable Parallel Programming with CUDA
  6. Parallel Computing Experiences with CUDA
  7. Roofline: An Insightful Visual Performance Model for Multicore Architectures
  8. Bringing UMAP Closer to the Speed of Light with GPU Acceleration
  9. GPU-Accelerated Single-Cell Analysis at Scale with rapids-singlecell

Author Profile

Miguel Salas
Miguel Salas
I am physicist and electrical engineer. My knowledge in computer software and hardware stems for my years spent doing research in optics and photonics devices and running simulations through various programming languages. My goal was to work for the quantum computing research team at IBM but Im now working with Astrophysical Simulations through Python. Most of the science related posts are written by me, the rest have different authors but I edited the final versions to fit the site's format.

Miguel Salas

I am physicist and electrical engineer. My knowledge in computer software and hardware stems for my years spent doing research in optics and photonics devices and running simulations through various programming languages. My goal was to work for the quantum computing research team at IBM but Im now working with Astrophysical Simulations through Python. Most of the science related posts are written by me, the rest have different authors but I edited the final versions to fit the site's format.

Leave a Reply

Your email address will not be published. Required fields are marked *