So what’s the big deal?

Whether it’s used to audit fees or conversions, to shed light on auction dynamics, or feed robust bidding optimizations and algorithms, log- or event-level data is easy to ask for, but harder to use effectively.

Thankfully this is adtech, where number-crunching and terabytes of data are par for the course and companies typically invest in their own business intelligence teams.

Determined to realize the benefits of supply path optimizations (SPO) and for greater transparency regarding ‘taxes’ and fees in the supply chain, keen brands and agencies have started asking for auction-level data that can be attributed to a single impression. 

How does it work?

A row or log for each event is valuable for machine learning which needs 1,000s or 1,000,000s of data to learn from, it’s valuable for audit-like purposes, etc. An event could be a bid request, response, auction, impression, or conversion and they may include a common id to relate a request to an impression or auction. This is different from traditional reporting in which all requests with common dimensions would be grouped together, or aggregated. Even a highly granular report would not be considered log-level. 

However, log-level data is not raw data—it is still delivered as a csv or other file type and isn’t a truly unadulterated json file. Raw data as it is truly sent to a buyer or seller is more useful for troubleshooting than machine learning.

How is the data shared?

The data is not delivered as a standard report, rather it is shared in data ‘buckets’ such as Amazon S3 belonging to the customer. Amazon Simple Storage Service (S3) is a data storage service that stores objects consisting of data and its descriptive metadata. It is relatively inexpensive, easy-to-use and to scale. 

The data is either batched or shared in real-time and in either case is sometimes aggregated minimally. Batched data means that data is being processed in batch rather than in real-time. The system waits for a certain amount of time or amount of data to arrive before starting to process it. Then at process time the system decides whether to aggregate or not. Batching and aggregation are two different stages of data processing.

Some exchanges batch the data hourly or daily, while others share it in real-time with no delays. This depends upon the exchange, their infrastructure and typical agreement between the exchange and the customer.

What is data interoperability?

Where logs originate can impact their efficacy—since this is not something with industry standardization, different exchanges and DSPs often have different data dictionaries, ways of defining log-level vs aggregate, and of course, different ways to implement the findings such as an improved bidding algorithm. 

Advertisers and agencies have traditionally leveraged the DSP report and logs. While it makes it easy to use that DSP to optimize, it also locks a buyer to that technology and makes it difficult to switch. One solution is to get the data from another source like an SSP, that would also offer a fuller picture of success and lost bid. Another is to develop your own data definitions and preemptively align it with both the data and controls offered by any buying platform you may way to use. Here, it’s important to assess how often the available data changes—are there 20 release notes per year that change the attributes included? Or can you expect stability over a 3 year timeline? Do the available bid optimization controls line up with the dimensions you are adding to your optimization engine? Are these likely to change?

This can all be mitigated by using a common set of attributes and calculations, and increasing the interoperability of data. To be interoperable, a product—in this case the data—must have clear interfaces and be able to work with other systems without restrictions. More specifically, a buyer should demand that their data is portable, that the dimensions are not specific to the system sending it to them, and in general that it is easy to use anywhere. 

With the growing availability of log-level data, better understanding of bidstream dimensions, and a fresh approach to broken addressability and attribution workflows, we expect to see more innovative use cases for log-level data.