Understanding Benchmarking

As a Tangle Network operator, you should understand how the network benchmarks your system to determine pricing for blueprints. This guide explains the automated benchmarking process and how it affects the quotes generated for your operator runtime.

What is Blueprint Benchmarking?

Blueprint benchmarking is an automated process that measures your system’s capabilities to determine:

The cost to run specific blueprints on your operator host
The resource allocation for different blueprint types
Your operator’s competitive position in the operator marketplace

When users request quotes from your operator, Tangle’s pricing engine uses these benchmark results to calculate fair prices based on your hardware profile.

How Benchmarking Works

The benchmarking process happens automatically in two key phases:

1. On Service Activation (Pricing Engine)

When the pricing engine sees a ServiceActivated event for a blueprint, it runs a benchmark to refresh the local hardware profile for that blueprint:

The pricing engine listens to ITangle events via the Tangle EVM RPC.
A benchmark task runs and caches CPU/memory/storage/network measurements.
The cached profile is keyed by blueprint ID and used for future quote requests.

2. During Blueprint Runtime (QoS + Metrics)

While blueprints run on your operator host, runtime metrics can be collected for observability:

The Blueprint Manager can emit QoS metrics and heartbeats.
Dashboards may display resource usage and job-level stats.
These metrics are for visibility; the pricing engine still relies on its local benchmark cache.

Resources That Are Automatically Measured

The benchmarking system automatically collects metrics on these key resources:

Resource	What’s Measured Automatically	Why It Matters
CPU	Utilization, Available cores, Performance	Primary factor in blueprint execution speed
Memory	Available RAM, Usage patterns	Determines capacity for complex operations
Storage	Available space, Read/write speeds	Affects data processing capabilities
Network	Connection quality, Bandwidth	Impacts communication performance
GPU	Utilization, Memory (when applicable)	Important for specialized workloads

These metrics are relevant to the Pricing Configuration used by the network:

# Minimal example of how metrics map to pricing
[blueprint.resources]
cpu = { count = 8, price_per_unit = "0.001" }
memory = { count = 16384, price_per_unit = "0.00005" }
storage = { count = 1024000, price_per_unit = "0.00002" }

How Benchmarking Affects Your Operator’s Quotes

The benchmark results directly influence how the pricing engine generates quotes:

When a user requests a blueprint quote from your operator, the pricing engine retrieves the cached benchmark profile
It applies the blueprint’s resource requirements to your profile
It calculates the quote using this formula:

Quote = Base Resource Cost × Time Multiplier × Security Commitment Factor

Where:

Base Resource Cost: Derived from your operator benchmarking results
Time Multiplier: Adjusts cost based on how long the service will run
Security Commitment Factor: Based on the asset security commitments required by the request

Example Scenario

For example, when a user requests a blueprint using 2 CPU cores, 4GB RAM, and 50GB storage for 24 hours, the pricing engine might calculate the following (note that these values are examples and do not necessarily reflect real costs):

CPU: 2 cores × $0.00001 per core-second × 24 hours (in blocks) = $4.32
Memory: 4096 MB × $0.000005 per MB-second × 24 hours (in blocks) = $1.024
Storage: 51200 MB × $0.000002 per MB-second × 24 hours (in blocks) = $0.512

Base Resource Cost = $6.256

The final quote would then be adjusted based on network conditions and security requirements.

Viewing Benchmark Information

Benchmark results are stored in the pricing engine’s local cache and surfaced via logs/metrics. QoS dashboards are blueprint-specific and depend on what the operator exposes.

Frequently Asked Questions

Q: Do I need to manually run benchmarks?
A: No, the pricing engine runs benchmarks automatically when services activate and refreshes its cache as needed.

Q: Can I improve my benchmark scores?
A: While you can’t directly modify the benchmarking process, upgrading your hardware or optimizing your system can indirectly improve your operator performance.

Q: How often are benchmarks updated?
A: Benchmarks are initially created during registration and may be updated periodically or when significant system changes are detected.

Q: Do benchmark results affect which jobs I receive?
A: Yes, users may choose operators based partly on performance metrics derived from benchmarks.

To learn more about operating on the Tangle Network, you may want to review:

Understanding the benchmarking process helps you better appreciate how the Tangle Network determines pricing for blueprints running on your operator.

Pricing Basics Quality of Service