{"id":15002,"date":"2026-03-08T08:34:23","date_gmt":"2026-03-08T07:34:23","guid":{"rendered":"https:\/\/emilkirkegaard.dk\/en\/?p=15002"},"modified":"2026-03-08T08:34:23","modified_gmt":"2026-03-08T07:34:23","slug":"recency-bias-in-imdb-ratings","status":"publish","type":"post","link":"https:\/\/emilkirkegaard.dk\/en\/2026\/03\/recency-bias-in-imdb-ratings\/","title":{"rendered":"Recency bias in IMDB ratings"},"content":{"rendered":"

In 2024 I wrote a post showing this plot<\/a> (well, a similar one):<\/p>\n

\"\"<\/a><\/p>\n

The pattern is that newer movies have been getting progressively lower rated. Now there can be some reasons for this pattern:<\/p>\n

    \n
  1. Movies are just getting worse over time because of inherent lowering of movie production skills or cultural decline and ratings accurately reflect this.<\/li>\n
  2. Making movies used to be very expensive and most people couldn’t do it, that is, the barrier to entry was much higher, and because of this, only the relatively more plausible movie projects got filmed and the ratings reflect this.<\/li>\n
  3. Same as the above, but also between countries, where there are a lot of bad movies produced also in Bollywood, Nollywood and other places.<\/li>\n
  4. There is a lower coverage % of movies for older years, since few people bother adding some C-tier movie from 1930 to the IMDB database.<\/li>\n<\/ol>\n

    The above pattern is not due to short movies or porn, which while it exists in IMDB’s database, I have filtered them out. Here’s the decline\u00a0 pattern by major regions (IMDB dataset does not have a main production country or studio country, but has various indirect methods):<\/p>\n

    \"\"<\/a><\/p>\n

    There is roughly a -0.01 decline most places, but no decline seen in Japan and South Korea. Whatever is responsble for this trend, it is nearly global. Nor can the decline be attributed entirely to the infusion of low quality 3rd world films, however, non-US movies pull the overall slope down by ~23% relative to the US-only slope, so that explanation was partially true. Insofar as Big Woke is destroying American cinema, it seems the exact opposite is the case with the global trends — America is making the decline weaker.<\/p>\n

    Anyway, the point of this post was to look more closely at the apparent recent uptick in movie ratings. Again, this could reflect a genuine sudden increase in quality, but it is more likely to be a kind of movie rater selection bias, where people who are really excited about new movie X go to the cinemas or otherwise see it before other people do. Call it the fanboi effect. If we assume this uptick just reflects bias, we can estimate the effect size of this bias using linear regression with dummies for recent years with cross sectional data (1 time point). However, if we want to be sure it really is a temporary effect, we need data for the same movies over time. Sadly, the public IMDB datasets<\/a> do not provide such longitudinal data, nor even a distribution of the ratings for a given movie (% of 10\/10 etc.). What we can do instead is to … download their rating file every day for some years and then analyze that data. This way, we can see how ratings change as movie get more ratings or age in general. Specifically, we can see this for all movie ages, as a movie made in 2000 was already 20+ years old and just got X years older in the period under study. Now maybe you don’t have patience to remember to download these files every month for years. But a computer does. So I had a script running on a server to download these files every day for the last 2 years or so. But it turns out I didn’t need bother with this because the Internet Archive covers the page since 2018<\/a>. There isn’t data for every day, but there are some ~1200 files, of which I downloaded about 2 per month (sadly, this means my 2 year scraping project was in vain!). Using the within movie variation, we can estimate the recency bias quite accurately:<\/p>\n

    \"\"<\/a><\/p>\n

    The largest effect is seen early on, of course, in line with the cross-sectional plot. The different intercepts show the cohort decline effect. We can compare the effect sizes from the two methods:<\/p>\n

    \"\"<\/a><\/p>\n

    The results are reasonably but not entirely consistent. Movie age 6 is an outlier. This is peak COVID with cinema lockdowns (2021). Probably it reflects that bigger studios delayed releasing their best movies so as to not forego profits, thus causing a temporary downward spike. We can also confirm the mechanism using changes in the number of votes for a movie:<\/p>\n

    \"\"<\/a><\/p>\n

    Note that we cannot just rely on number of votes at a single time of time because people prefer watching the same good movies, thus there is a positive correlation between number of votes and the ratings:<\/p>\n

    \"\"<\/a><\/p>\n

    It’s actually a Simpson’s paradox of sorts. The effect of number of votes is overall positive, but not within movies, where it is initially negative and eventually neutral.<\/p>\n

    Using these cohort and age effects, it is possible to adjust a given movie to its likely long-term rating, and also compute a kind of movie IQ score, that is, cohort relative score. This is mostly just for fun:<\/p>\n

    \"\"<\/a><\/p>\n

    Or a different way:<\/p>\n

    \"\"<\/a><\/p>\n

    Supposing you wanted to watch top 2% (130+) movies, even limiting yourself to those with 8.5 ratings is not enough if it is a recent movie, as this only corresponds to 126 movie IQ or so (even less for 2026).<\/p>\n","protected":false},"excerpt":{"rendered":"

    In 2024 I wrote a post showing this plot (well, a similar one): The pattern is that newer movies have been getting progressively lower rated. Now there can be some reasons for this pattern: Movies are just getting worse over time because of inherent lowering of movie production skills or cultural decline and ratings accurately […]<\/p>\n","protected":false},"author":17,"featured_media":15005,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1107],"tags":[3460,3278,3694],"class_list":["post-15002","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-science","tag-imdb","tag-movies","tag-ratings","entry","has-media"],"_links":{"self":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/15002","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/users\/17"}],"replies":[{"embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/comments?post=15002"}],"version-history":[{"count":1,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/15002\/revisions"}],"predecessor-version":[{"id":15013,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/15002\/revisions\/15013"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/media\/15005"}],"wp:attachment":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/media?parent=15002"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/categories?post=15002"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/tags?post=15002"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}