Project
An open and extensible framework for building observability toolkits and instruments that emit observations over a circuit of contexts (sources) and conduits (nodes) via currents (flows). The Substrates project is the reference implementation for the Humainary Substrates API, an API employed across all observability software libraries under the scope of the Humainary initiative.
Posts
- From Many to One Observability Model
Today the approach to observability at various stages within the data pipeline, from Application to Console, has been to create different models in terms of concepts, structure, features, and values. But what if the model employed were the same across all stages in a data pipeline? - Multi-Event-Casting with Inlets
The typical flow of execution for an observability Instrument is for instrumentation within the application code to make a single call to a method defined in the Instrument interface. But there are cases where a single call into an Instrument interface causes the dispatching of multiple events. - Scaling Observability Resources
A significant challenge in building observability agents or heavily instrumented applications or frameworks is in scaling, both up and down, the resources consumed. There is a trade-off here in terms of time and cost. - Composing Instrument Circuits
In this post, we walk through one of the showcases in the project’s code repository that demonstrates how the complexity of hooking up components in a Circuit is greatly simplified. - Designing for Extensibility
The interfaces defined within Substrates API are designed with extensibility and evolution in mind, both from a client library and provider implementation perspective. - Pipelines and Projections
An objective of the Substrates API is that it should be possible for developers to be location independent in that code can execute without change within the local process itself or a remote server that is being served the data via a Relay. - Simplifying Instrument Synthesis
Three overriding principles are applied in the design of the Substrates API – consistency (standardizing), conciseness (simplifying), and correctness (sequencing). - API Design – A Moving Target
Good design takes time, over many iterations (of converging and diverging design paths), in developing, discovering, discussing, discounting, and occasionally destroying. - Data Pipeline Segmentation
The stages within a local data pipeline managed by the Substrates runtime are detailed, from a client call to an Instrument interface method to dispatching an Event to an Outlet registered by a Subscriber. - Playing with Pipelines
A walk through of one of the Substrates showcase examples hosted on GitHub, demonstrating two of the most critical aspects of pipelines – the production and consumption of captured data and published events. - Pipelines Reimagined
Using Substrates, the fusion of multiple streams of data from multiple sources, an essential process of any monitoring and management solution, can be done in-process and in real-time. - Circuits, not Pipelines!
The first official release of the Substrates API is nearing completion. With that, it is time to explore one of the toolkit’s most significant differences compared to other approaches, such as OpenTelemetry.