spectator-cpp Usage¶
CPP thin-client metrics library for use with Atlas and SpectatorD.
Supported CPP Versions¶
This library currently utilizes C++ 20.
Installing & Building¶
If your project uses CMake, you can easily integrate this library by calling add_subdirectory()
on the root folder. To build the Spectator-CPP thin client independently, follow the Docker
container instructions at Dockerfiles.
The container provides a minimal build environment with g++-13, python3, and conan. Spectator-CPP
relies on just three external dependencies (spdlog, gtest, and boost), which are managed automatically
via conan.
Instrumenting Code¶
#include <registry.h>
int main()
{
// Create common tags to be applied to all metrics sent to Atlas
std::unordered_map<std::string, std::string> commonTags{{"platform", "my-platform"}, {"process", "my-process"}};
// Create a config which defines the way you send metrics to SpectatorD
auto config = Config(WriterConfig(WriterTypes::UDP), commonTags);
// Initialize the Registry with the Config (Always required before sending metrics)
auto r = Registry(config);
// Create some meters
auto threadGauge = r.CreateGauge("threads");
auto queueGauge = r.CreateGauge("queue-size", {{"my-tags", "bar"}});
threadGauge.Set(GetNumThreads());
queueGauge.Set(GetQueueSize());
/* Metrics Sent:
"g:threads,platform=my-platform,process=my-process:5.000000\n"
"g:queue-size,my-tags=bar,platform=my-platform,process=my-process:10.000000\n"
*/
}
Logging¶
Logging uses the spdlog
library and outputs to standard output by default, with a default log
level of spdlog::level::info
. The Logger
class is a singleton and provides the
Logger::GetLogger()
function to access the logger instance. To change the log level, call
Logger::GetLogger()->set_level(spdlog::level::debug);
after the logger has been successfully
created.
Working with MeterId Objects¶
Each metric stored in Atlas is uniquely identified by the combination of the name and the tags
associated with it. In spectator-cpp
, this data is represented with MeterId
objects, created
by the Registry
. The CreateNewId()
method returns new a MeterId
object, which has extra
common tags applied, and which can be further customized by calling the WithTag()
and
WithTags()
methods. Each MeterId
will create and store a validated subset of the spectatord
protocol line to be written for each Meter
, when it is instantiated. Manipulating the tags with
the provided methods will create new MeterId
objects.
Note that all tag keys and values must be strings. Tags are represented as
std::unordered_map<std::string, std::string>
. For example, to track the number of successful
requests, ensure your tags use string values, such as {"statusCode": std::to_string(200)}
. The
MeterId
class validates all provided tag keys and values: if either is empty or contains only
whitespace, it will be dropped, and any invalid characters will be replaced with an underscore.
#include <registry.h>
int main()
{
// Create common tags
std::unordered_map<std::string, std::string> commonTags{{"platform", "my-platform"}, {"process", "my-process"}};
// Initialize the Registry
auto config = Config(WriterConfig(WriterTypes::UDP), commonTags);
auto registry = Registry(config);
// Option 1: Using the registry to create a MeterId & creating a Counter from the MeterId
auto numRequestsId = registry.CreateNewId("server.numRequests", {{"statusCode", std::to_string(200)}});
registry.CreateCounter(numRequestsId).Increment();
// Option 2: Directly creating a Counter
auto numRequestsCounter = registry.CreateCounter("server.numRequests2", {{"statusCode", std::to_string(200)}});
numRequestsCounter.Increment();
}
Atlas metrics will be consumed by users many times after the data has been reported, so they should be chosen thoughtfully, while considering how they will be used. See the naming conventions page for general guidelines on metrics naming and restrictions.
Meter Types¶
- Age Gauge
- Counter
- Distribution Summary
- Gauge
- Max Gauge
- Monotonic Counter
- Monotonic Counter Uint
- Percentile Distribution Summary
- Percentile Timer
- Timer
Output Locations¶
spectator.Registry
supports three output writer types: Memory Writer, UDP Writer, and Unix Domain
Socket (UDS) Writer. To specify the writer type, initialize the Registry with a Config
object. A
Config
requires a WriterConfig
(which defines the writer type and location) and can optionally
include extra tags to be applied to all metrics. The WriterConfig
also accepts an optional
buffer size parameter, enabling buffering for all writer types.
Writer Config Constructors¶
// Constructor 1: Define a location and no buffering
WriterConfig(const std::string& type)
// Constructor 2: Define a location with buffering
WriterConfig(const std::string& type, unsigned int bufferSize);
Writer Config Examples¶
/* Default Writer Config Examples */
// Write metrics to memory for testing
WriterConfig wConfig(WriterTypes::Memory);
// Default UDP address for spectatord
WriterConfig wConfig(WriterTypes::UDP);
// Default UDS address for spectatord with buffering
WriterConfig wConfig(WriterTypes::Unix, 4096);
/* Custom Writer Config Location Examples */
// Custom UDP writer location
std::string udpUrl = std::string(WriterTypes::UDPURL) + "192.168.1.100:8125";
WriterConfig wConfig(udpUrl);
// Custom UDS writer location
std::string unixUrl = std::string(WriterTypes::UnixURL) + "/var/run/custom/socket.sock";
WriterConfig wConfig(unixUrl, 4096);
Config Constructor¶
// Constructor: WriterConfig & optional extraTags for all metrics
Config(const WriterConfig& writerConfig, const std::unordered_map<std::string, std::string>& extraTags = {});
Config Examples¶
/* Config Examples */
// Config with a WriterConfig & no extra tags
Config config = Config(WriterConfig(WriterTypes::Memory));
// Config with a WriterConfig & extra tags
std::unordered_map<std::string, std::string> commonTags{{"platform", "my-platform"}, {"process", "my-process"}};
Config config = Config(WriterConfig(WriterTypes::Memory), commonTags);
/* Registry Initialization */
WriterConfig wConfig(WriterTypes::Memory);
Config config = Config(wConfig);
Registry registry(config);
Location can also be set through the environment variable SPECTATOR_OUTPUT_LOCATION
. If both are
set, the environment variable takes precedence over the value passed to the WriterConfig. If either
values provided to the WriterConfig are invalid, then a runtime exception will be thrown.
Line Buffer¶
The WriterConfig
allows you to set an optional bufferSize
parameter. If bufferSize
is not
set, each metric is sent immediately to spectatord
using the configured writer type. If
bufferSize
is set, metrics are buffered locally and only flushed to spectatord
when the buffer
exceeds the specified size. For high-performance scenarios, a buffer size of 60KB is recommended.
The maximum buffer size for UDP and Unix Domain Socket writers on Linux is 64KB, so ensure your
buffer size does not exceed this limit.
Batch Usage¶
When using spectator-cpp
to report metrics from a batch job, ensure that the batch job runs for at
least five (5), if not ten (10) seconds in duration. This is necessary in order to allow sufficient
time for spectatord
to publish metrics to the Atlas backend; it publishes every five seconds. If
your job does not run this long, or you find you are missing metrics that were reported at the end
of your job run, then add a five-second sleep before exiting. This will allow time for the metrics
to be sent.
Debug Metrics Delivery to spectatord
¶
In order to see debug log messages from spectatord
, create an /etc/default/spectatord
file with
the following contents:
SPECTATORD_OPTIONS="--verbose"
This will report all metrics that are sent to the Atlas backend in the spectatord
logs, which will
provide an opportunity to correlate metrics publishing events from your client code.
Design Considerations - Reporting Intervals¶
This client is stateless, and sends a UDP packet (or unixgram) to spectatord
each time a meter is
updated. If you are performing high-volume operations, on the order of tens-of-thousands or millions
of operations per second, then you should pre-aggregate your metrics and report them at a cadence
closer to the spectatord
publish interval of 5 seconds. This will keep the CPU usage related to
spectator-cpp
and spectatord
low (around 1% or less), as compared to up to 40% for high-volume
scenarios. If you choose to use the WriterConfig
with the buffering feature enabled, metrics will
only be sent when the buffer exceeds its size. The buffer is protected by a mutex, allowing
multiple threads to safely write metrics concurrently.
Writing Tests¶
To write tests against this library, instantiate a test instance of the Registry
and configure it
to use the MemoryWriter
, which stores all updates in a Vector
. Maintain a handle to the
MemoryWriter
, then inspect the protocol lines with GetMessages()
to verify your metric updates.
See the source code for more testing examples.
int main()
{
// Initialize Registry
auto config = Config(WriterConfig(WriterTypes::Memory));
auto registry = Registry(config);
// Directly create a Counter
auto numRequestsCounter = registry.CreateCounter("server.numRequests2", {{"statusCode", std::to_string(200)}});
// Create a handle to the Writer
auto memoryWriter = static_cast<MemoryWriter*>(WriterTestHelper::GetImpl());
numRequestsCounter.Increment();
auto messages = memoryWriter->GetMessages();
for (const auto& message : messages) {
std::cout << message; // Print all messages sent to SpectatorD
}
}
Performance¶
On an m5d.2xlarge
EC2 instance, with spectator-cpp-2.0
and
github.com/Netflix/spectator-cpp/v2 v2.0.0
, we have observed the following single-threaded
performance numbers across a two-minute test window (unbuffered scenario):
- 113,655.11 requests/second over
udp
- 132,490.97 requests/second over
unix
The benchmark incremented a single counter with two tags in a tight loop, to simulate real-world
tag usage, and the rate-per-second observed on the corresponding Atlas graph matched. The protocol
line was 74
characters in length.