Whether you are selling tickets to a live event, fruit from a roadside stand or products on the internet, measurability is everything. Without accurate data, you have no way to assess your results, let alone make projections. This is one of the many advantages to driving traffic from the internet.
Every click, view, and order placed is both tracked and can be used by you, the advertiser, in one way or another. This is great, except for when the tracking not only fails, but generates completely inaccurate results that wreak havoc on your account and cost you immeasurable amounts of money in the process…
Facebook/Instagram Ads Manager Purchase Reporting Inaccuracy
In Facebook Ads Manager reporting (shared with Instagram reporting), there can sometimes be lag and, ultimately, real and accurate results can take time to generate. With that said, I had a very odd day+ that looks to be impacting everything, in a negative way (primarily due to the poor reporting influence on any spend thereafter).
It is important to note that I am and have been using the FB pixel for over a year. Additionally, I track/have orders reporting via both the FB pixel/server side and the Conversions API. The status in Events Manager indicates “No issues detected.” I always monitor Events Manager for even the slightest errors, as this is where small issues can turn into big problems if you are not paying attention.
This would be the first place to check for any sort of erroneous event duplication and I have meticulously checked the logs one by one to ensure that there is no issue within Events Manager or the pixel itself.
First, let’s look at a normal day…
On 3/31/21, we had 362 unique purchases and 389 total purchases sent from FB Ads, as reported by Ads Manager. Now, I am looking at those numbers right now (late evening 4/4), so I assume we had some number of repeat buyers since those actual clicks and conversions took place four days ago, and so the 362 to 389 difference between unique and total purchases is not wild or unordinary.
Usually there is around 1-5% normal variance in total vs. unique purchases within, and on (meaning, checked the same day in reports) the same day, between random double orders, the occasional de-duplication omission, or otherwise imperfect reporting. This is all normal and to be expected. This particular day, checked 4+ days after, amounted to a 7% difference between unique and total purchase orders. This was, and is, normal and created no cause for any sort of concern.
Now, moving on to 4/1/21…
On this day we had 325 unique purchases and 442 total purchases. Yes, there are a few additional ad sets here (in the pic below), but they have no impact neither to the bottom line result nor the causation (one of the few things I am sure of!).
So, on 3/31, even with a longer lag since the clicks took place, there was only a 7% differential between unique and total purchases. On this day, however, with the discrepancy between 325 unique and 442 total purchases, there is a difference of 26.5%. This is a 378% increase in spread from just the day before. Now it’s clear there is most likely a problem. It is only one day, however, so it’s not a certainty, but this is cause for alarm barring a bonafide lag in reporting.
I checked the Events Manager log for purchase events on this day and there was no duplication. Additionally, Events Manager was almost 100% accurate in reporting all purchases sitewide that day, regardless of its source, as it should. Given pixel is the source for Events Manager, and in turn the log (my only way to confirm reporting accuracy on an event by event basis), this is where I began to have concern. But, with that said, I considered that a 1-2 day lag in FB reporting was possible and that true results might update in time.
I gave FB reporting even more leeway with consideration for the iOS 14 updates and the generally “changing” nature of the platform as a whole in 2021.
Then, I got to 4/2/21…
This is when things got even worse. By 4/2 everything was wildly off; to the point where I could only conclude that it either went off the rails somewhere mid-day 4/1 and ran into 4/2 or that it was somehow getting progressively worse as each day (and hour) passed by.
On 4/2 there was a total of 383 unique purchases and 619 total purchases. On top of this discrepancy, I (having been watching this situation in real-time), anecdotally noticed that the total purchases column was updating/adding orders almost by the minute into the morning hours of 4/3, well after the day had ended, but that there, in tandem, was little to no update to the unique purchases column.
The difference between unique and total orders on 4/2 was 38.2%. In the span of a few days, the differential between unique and total purchases grew from 7% to 26.5% to 38.2%. At this point I did not see the prior days updating/correcting or otherwise self-resolving–and I knew something was wrong.
I only looked at 4/2 reporting, in its entirety, a day later–on 4/3–because I wanted to give everything time to settle. Nothing changed positively, but by that time in the day, I was able to also see same-day results for 4/3; then things got even more weird…
Next, we get to 4/3/21…
I am glad that I did not react quickly and pause everything, because 4/3 ended up making everything “better” while also creating even further confusion. On 4/2 into 4/3 I did reduce some bids to help mitigate the bleeding caused by poor data and inefficient bidding that was starting to clearly take place as a result of the inaccurate purchase results. With that said, the bid adjustments had no bearing on the reporting accuracy for purchase events, and the results were the most interesting part…
4/3/21 ended up sending 371 unique purchase events and 389 total purchases. This ends up amounting to only a 4.7% difference in unique vs. total purchases. On one hand, this was great to see. On the other, we had wasted a lot of spend that resulted from the bad reporting (and in turn fed the AI bad information to bid with). I have started to see (as of end of day 4/4) that the bidding/AI has started to rollback its execution that was predicated on the very wrong purchase event reporting from the days before.
So, now the question is…if nothing was changed, before, during, or after these days–on our site, with the pixel, or in any other way–that would impact events reporting accuracy, why did it happen at all? I am happy to see that the issue has seemingly fixed itself, but I would still like to know what happened; primarily as a preventative measure, but also for peace of mind.
I have done every bit of data analysis and taken apart every piece of information that I possibly can in an effort to see what may have caused this very odd misreporting, but I can not find anything sensical, let alone definitive, and the account self-resolving only makes the situation even more mysterious. I hope that I will be able to share an update, and ideally a positive one, very soon…