US discloses error in college earnings data, says overall impact was limited

The US Department of Education has disclosed an error in some earnings data published through its College Scorecard, after discovering that graduate-level students had been inadvertently included in calculations intended to measure undergraduate outcomes.

In a data errata document dated Oct 15, the department said the error affected four-year, post-completion earnings calculations in the 2022–23 data file.

These calculations covered undergraduate students who completed their studies in the 2017–18 and 2018–19 academic years, using earnings data for calendar years 2022 and 2023, respectively.

The affected figures were included in the College Scorecard's downloadable data files and consumer-facing website. They were also presented on the lower-earnings summary page of the Free Application for Federal Student Aid (FAFSA).

How many institutions were affected?

The coding error affected institutions offering both undergraduate and graduate programmes. Some graduate-level completers were incorrectly counted in undergraduate earnings cohorts. Some institutions offering only graduate programmes also had non-null values for undergraduate earnings metrics.

Institutions that do not offer graduate programmes were unaffected by this particular coding error. The department also confirmed that programme-level calculations were not affected.

Among 1,553 institutions offering both undergraduate and graduate programmes, the median share of graduate-level completers inadvertently included in the earnings cohorts was 8.9%. The average was 13.3%, with the figures varying considerably across institutions.

The department's analysis found that the proportion of graduate students included was 3.9% or less at the 25th percentile of institutions and 16.3% or more at the 75th percentile. At the 95th percentile, the share reached 44.3%.

Department says error had no meaningful overall impact

To assess the impact, the department compared the previously published four-year earnings calculations in the 2022–23 data file with newly published five-year calculations using an improved methodology.

Both sets of calculations measure earnings in calendar years 2022 and 2023, although they use different completion cohorts.

The median change between the four-year and five-year calculations was 1,103 among institutions offering graduate programmes and 1,212 among institutions without graduate programmes.

The difference between these figures was 109, equivalent to approximately 2.5% of the standard deviation across all institutions, which was 4,433.

The department said the distributions of changes were similar between the two groups. It also found that the interquartile ranges—the spread of the middle 50% of observations — differed by only 82.

Based on these comparisons, the department concluded that there was no evidence that the inclusion error had made a meaningful difference to the overall distribution of earnings outcomes.

This is the department's assessment of the error's impact; it does not mean that every affected institution's previously published figure was accurate.

Other limitations remain in institution-level data

The department noted that other institution-level undergraduate earnings calculations may also include some graduate-level students because of limitations in the enrolment records used to identify students' study levels.

Graduate students are generally identified and excluded when they received federal Title IV financial aid at the graduate level. However, students who previously received Title IV aid as undergraduates may be mistakenly included in undergraduate calculations if they later attend the same institution as graduate students without receiving Title IV aid at that level.

To address these limitations, the department said its new calculations in the 2022–23 data file—apart from the affected four-year measurements—use programme-level completion records to distinguish undergraduate and graduate students.

What should applicants check

For Indian students comparing US universities, the disclosure highlights the importance of checking what an earnings figure actually measures before using it to assess an institution.

Applicants should distinguish institution-level outcomes from programme-level data and check the cohort and time period behind the figures.

The department's disclosure does not establish that all College Scorecard earnings data are unreliable, nor does it say that every institution offering graduate programmes had the same degree of error. Its findings concern a specific set of institution-level calculations and the department's assessment of their overall impact.