Date range comparison bias
When making decisions based on data, we want to guard against bias. Both the people making decisions and the data itself can be biased. Even when people & data are not biased – another type of usually inadvertent bias is still possible: date range comparison bias.
Date range comparison bias happens when the current date range, and the “compare to” previous date range, are dissimilar, and lead to incorrect conclusions. A crude example would be comparing the last week to the previous month. The more problematic instances of this are nuanced. You can’t fix a problem if you do not know it is there. The following best practices will help prevent such nuanced bias.
Date range comparison best practices
- Never include the current day in your date range. The “compare to” earlier range will have all days with a full set of recorded data, whereas the current day will have partial data.
- Generally, use a seven (7) day minimum in each comparison date range. This will smooth out anomalies that can happen on a given day in a week.
- Use the same day of week start & end for both periods. Weekdays and weekends can have very different behavior. Also, if behavior related to the measured data is very different at the start or end of the month, give this consideration as well when selecting your comparison range (e.g., make sure both comparison periods include a like end of month or beginning of month instance).
- Don’t simply compare one month to another month. Each month is distinct in either the number of days it covers, or the day of week at the start & end, or in the number of weekends it includes. If, for example, April just started, and you are doing March reporting, don’t just compare March to February. March has 31 days, and February has 28 or 29 days. Instead, consider comparable 28 day periods. Selecting the last 28 days, and comparing to the previous period, will automatically assign like day of week to the start & end of both periods, although you may still need to adjust for start & end of the month as well.
- Consider the impact of holidays on your date range comparison. There is obvious bias introduced, for example, if one date range includes the week of Thanksgiving and one does not.
COVID-19 date range comparison considerations
For date ranges with data influenced by COVID-19, significant changes happen quickly. Know that data in week 1 will have a very different COVID-19 impact applied to it than week 4. This is obviously true when comparing a COVID-19 impacted date range to a pre-COVID-19 time period. Consider this phenomenon in your date ranges for analysis, and certainly when selecting date range comparisons. As we noted in our COVID-19 impact on actionable analysis post, we think March 11-12, 2020, represents the start of COVID-19 meaningful impact on consumer & business behavior.
Date range comparisons in Google Analytics properties
You’ll notice when you select and compare date ranges in a standard Google Analytics property vs. the new App+Web property, they are significantly different. The new App+Web property date range picker is like that of Google Ads, including much more flexible and useful base + comparison date range options.
Per our comments in “Start tracking now with a Google Analytics App + Web property“, invest in standing up one of these new property types to collect data for your digital property as soon as possible. The tracking code for the new property type will not interfere with the standard property type code. This new property type is where Google is investing. It will eventually be the “go to” Google Analytics property type for analysis.
The sooner you start collecting data in this property type, the more history you’ll have in it for analysis, and the sooner you can put the new date range picker to work helping you guard against date range comparison bias.
