Examining the ADL’s Antisemitism Audit
Robin Stevens Brody / Sipa USA via AP
A line-by-line reassessment of the organization’s data illuminates the flaws in its methodology.
by Shane Burley and Jonah ben Avraham
IN MID-APRIL, the Anti-Defamation League (ADL) released a report tallying the total number of antisemitic incidents in the United States in 2023. The annual audit highlights 8,873 instances of antisemitic “harassment, vandalism and assault”—a 140% increase from the organization’s 2022 numbers, and the highest since the organization began tracking such incidents in 1979. The report was quickly picked up by Axios, The Seattle Times, and various local TV stations, among other outlets, and often uncritically cited as evidence that the movement for a ceasefire in Gaza is a wellspring of rising antisemitism. “The ADL . . . tracked 1,352 anti-Israel rallies where antisemitic and anti-Zionist rhetoric was observed after October 7,” wrote Nicole Chavez in CNN’s summary of the report, which also focused on the ADL’s count of 922 allegedly antisemitic incidents on college campuses.
The ADL has not been shy about pushing this narrative: Just after it released the audit, the organization joined calls by congressional Republicans for law enforcement to shut down campus Gaza solidarity encampments. “President Shafik must invite NYPD back on campus or Governor Kathy Hochul should direct the National Guard to protect our kids,” ADL CEO Jonathan Greenblatt posted on X in late April in response to Columbia’s ongoing encampment. Greenblatt and the ADL have insisted that the encampments are hotbeds of antisemitic vitriol, undermining Jewish safety and building on an unprecedented spike in antisemitism since October 7th—a spike they say is substantiated in statistical reports like the audit.
But considering that the ADL has recently attracted scrutiny over its ongoing effort to categorize anti-Zionism as antisemitism—and has historically faced broader questions over its statistical methods—it is unclear whether the organization’s audit provides an accurate portrait of American antisemitism. So we reanalyzed the ADL’s data line-by-line, drawing on our expertise as researchers and writers on contemporary antisemitism. In the audit, the ADL claims that it is “careful not to conflate general criticism of Israel or anti-Israel activism with antisemitism” but specifies that its “approach to Israel-related expressions comports with the IHRA [International Holocaust Remembrance Alliance] definition of antisemitism.” But it is precisely this definition that critics argue conflates legitimate criticisms of Israel or Zionism with antisemitism. To assess each entry and determine whether to classify it as an antisemitic incident, we instead used the Jerusalem Declaration on Antisemitism (JDA) definition, a tool developed by scholars to identify antisemitism while avoiding erroneous conflations of anti-Zionism with antisemitism.
While we expected that this difference in methodology would create a disparity between our findings and the ADL’s, our reappraisal also highlighted more basic problems with the ADL’s tracking system. In addition to identifying more than a thousand items we believe were misclassified as antisemitic—all cases of speech critical of Israel or Zionism—we found that the data included misapplications of the organization’s own standards and often did not provide enough information for us to assess the group’s judgment. Our analysis clarifies what the ADL’s prominent report captures and excludes, and shows how the conflation of anti-Zionism and antisemitism skews the data—ultimately serving as a reminder of the need for serious statistical analysis done by an organization not beholden to Israel advocacy.
In addition to identifying more than a thousand items we believe were misclassified as antisemitic, we found that the data included misapplications of the organization’s own standards and often did not provide enough information for us to assess the group’s judgment.
THE ADL’S YEARLY AUDITS have long been compiled from reports by private individuals and data sent by law enforcement; since 2021, they have also included data from Jewish federations, synagogues, Hillel International, and other groups through a partnership with the Secure Community Network. All of these cases are reviewed by the ADL’s Center on Extremism, and those that are deemed credible and pertinent are collected on the ADL’s Hate, Extremism, Antisemitism, and Terrorism Map (H.E.A.T. Map), and placed into one or more of seven categories—Antisemitic Incidents, Anti-LGBTQ Incidents, White Supremacist Propaganda, White Supremacist Events, Extremist Murders, Terrorist Plots and Attacks, and Extremist/Police Shootouts. In 2023, the H.E.A.T Map tracked a total of 16,040 incidents, and 8,873 specifically antisemitic ones, which are summarized in the annual audit. (The ADL does exclude many incidents through its vetting process: This year, more than 15,000 reports of antisemitism were made directly to the ADL, a spokesperson told Jewish Currents.)
In reassessing those 8,873 antisemitic incidents using the JDA, we found that 4,948—a little over half—were straightforwardly antisemitic according to the facts provided by the ADL. These included more than a thousand bomb threats against synagogues and Jewish organizations and hundreds of cases of interpersonal antisemitic comments or slurs. One hundred forty-one of these cases counted as “assault,” meaning physical attacks on Jews accompanied by antisemitic language or other indicators of antisemitism. For example, on December 6th in Manhattan, someone punched a “visibly Jewish person” and said “you f—ing Jew.” On October 1st in Washington DC, another Jewish person had their kippah knocked off and was physically blocked on their way home from a Sukkot service. Most of the remaining thousands of antisemitic incidents were less severe, involving unsettling public speech or online messages. At a December 13th school board meeting in Teaneck, New Jersey, for instance, someone said, “I know the blasphemy of them who say they are Jews but are not but are the Synagogue of Satan.” Twenty percent of the incidents we determined to be antisemitic could be officially traced to white supremacist groups, usually featuring the distribution of leaflets, public protests, or flash mobs.
We also identified 1,175 separate incidents—about 13% of the total—better understood as antisemitic imagery that did not clearly target a Jewish person or institution, such as spray-painted swastikas under a highway overpass or the word “Jew” scrawled on an abandoned building. In one such case, in Swampscott, Massachusetts on December 2nd, someone noticed a swastika drawn in condensation on an empty restaurant table, which the ADL categorized as antisemitic “harassment.” Though Nazi imagery is obviously antisemitic and worthy of concern, we believe that such incidents are more likely to reflect obnoxious youth or edgelord behavior than consistent, deeply held hate for Jews, and therefore categorized them separately. Some of the swastikas reported in the data were scribbled alongside swear words or drawings of genitals, suggesting that the perpetrators may have been motivated by a desire to shock as much as by antisemitic attitudes. (Researchers, including those at the ADL, have long made these distinctions: A 2017 USA Today report, for example, found that antisemitism analysts distinguished between swastika “vandals,” who were “usually individuals, often kids rebelling against adult authority, seeking peer approval and probably hazy on what the symbol means except that it’s a ‘bad symbol used by bad guys’”—and “ideologues” from organized hate groups who, at the time of reporting, were increasingly eschewing the swastika because its notoriety affected their ability to seek mainstream appeal.) Without a better sense of the motivation behind such imagery, it is difficult to assess how much, if anything, it tells us about the scale or potential material impact of American antisemitism.
Here we also found occasional discrepancies in the ADL’s application of its own methodology. The audit itself claims that “swastikas are not included [as antisemitic incidents] in circumstances when they appear to be targeting a different minority group.” But when a swastika was tagged on a school wall in a Queens, New York, high school alongside the words “F— Palestine” on December 6th, it was listed as an antisemitic incident, as was swastika graffiti found in Blairstown, New Jersey, on March 18th paired with the n-word. Indeed, we found 19 incidents where Jews were clearly not the intended target of the categorized incident. When asked about these incidents, an ADL spokesperson told us that “[as] the symbol of the Nazi regime that carried out the Holocaust, swastikas have a unique ability to traumatize Jewish people, even if they are not the immediate target,” and it is therefore “important to track them in the audit.” While this is fair reasoning for including all swastika incidents, the ADL’s contradiction of its own methodology does not inspire confidence in the reliability of the data.
Our assessment of incidents diverged from the ADL’s most often when it came to analyzing pro-Palestine demonstrations. The ADL says that antisemitic incidents revolving around Israel or Zionism accounted for 3,162 incidents—about 36% of the total—and that 1,352 of these (about 15% of the total) involve the use of specific slogans that the ADL considers inherently antisemitic. The audit mentions the use of the phrase “from the river to the sea”—usually as part of the protest slogan “from the river to the sea, Palestine will be free”—as an example of antisemitism more than 600 times. “Respect existence or expect resistance” is listed as a form of antisemitic harassment in 21 separate incidents, and the slogan “when people are occupied, resistance is justified” is flagged 35 times. In all previous years, the ADL did not include pro-Palestine protests in its audits, but the group says in the audit that it has employed “new methodology” since October 7th that identifies language that expresses “opposition to Zionism” or is “perceived as supporting terrorism or attacks on Jews, Israelis or Zionists” as antisemitism. If signs or slogans using such language appear at a rally, the ADL includes the rally in its audit. This characterization assumes that all references to an occupied people’s resistance against their occupiers can only be motivated by ethnic hatred. (People may disagree about what constitutes legitimate ”resistance,” particularly in the context of armed militancy, but such debates have surfaced across history in varied political contexts. Militancy does not inherently reflect antisemitism.) In doing so, it inflates both the total number of antisemitic incidents and the share of them attributed to the Palestine movement. Our reappraisal using the JDA found that 1,472 such incidents—about 17% of the total incidents tracked—did not appear antisemitic. For 132 incidents, a bit over 1% of the total, we couldn’t definitively decide how to apply the JDA.
OUR REASSESSMENT also turned up many instances that could not be fully evaluated based on the information provided. In 498 cases—about 6% of the total—the ADL’s description of events did not provide enough information to assess whether or not an incident was antisemitic (this includes 77 incidents that were listed as “confidential,” with no details provided to the public). Because the audit presents incidents out of context, some are difficult to adequately assess. For example, on December 18th, non-kosher food was placed in the kosher section of an Orange, Connecticut supermarket. But no information is provided that might help us determine whether it was left there by accident or on purpose, and with what intention.
Since October 7th, we have seen frightening stories of antisemitism travel widely, only to be revealed as more complicated under greater scrutiny. In late October, images of Jewish students cowering in a Cooper Union library surrounded by pro-Palestine protesters banging on the glass went viral; this incident is included in the ADL’s dataset. But reporting later showed that circumstances were less black-and-white: The Jewish students weren’t trapped as had been claimed by outlets like The Washington Post and the The Times of Israel, and they were part of an earlier pro-Israel counterprotest, suggesting that the clash between them and the pro-Palestine demonstrators was political—not ethnic—in nature. People can debate whether this behavior by Palestine solidarity demonstrators was advisable, but the question for a report like the ADL’s should be whether these incidents are clearly antisemitic.
This problem of partial or decontextualized information often arises when the audit includes incidents of political conflict. For example, the audit includes dozens of cases of people tearing down posters of Israeli hostages held in Gaza. In several instances the act was accompanied by explicitly antisemitic imagery or rhetoric, like an October 22nd incident where someone ripped off a man’s kippah after tearing down the poster; in these cases, we also categorized the incidents as antisemitic. But most of the time, the data included no information other than that the posters were removed. It is possible that such an act shows willful disregard for Jewish suffering. But it is also the case that many such fliers were themselves posted as components of pro-Israel counter-demonstrations or in provocative locations, like in front of a Palestinian restaurant. (One version of the poster contains QR codes leading to the website for the pro-Israel activist group StandWithUs.) Many people, including Jews, who were filmed tearing them down have publicly explained that they were trying to counter a narrative that was being pushed to gain support for the bombing campaign in Gaza.
The ADL also frequently recorded incidents in which anti-Israel or anti-Zionist messages were directed at Jewish organizations, sometimes without specifying the organization in question. For example, on November 26th, the words “Israel, shame on you” were painted on the sidewalk in front of a Los Angeles “Jewish organization.” This act certainly could be antisemitic if the organization in question is an institution of regular Jewish life—like a synagogue, day school, or Jewish nonprofit—suggesting that the signage is directed at Jewish people as such, and thus that Jews are collectively responsible for Israel’s actions. But if the message is directed specifically at an advocate of Israeli state policy, it would not necessarily be antisemitic. The ADL audit methodology says they include anti-Israel messages if they target any non-political institution and only include incidents directed at a pro-Israel political organization if the content is straightforwardly antisemitic (for example, employing swastikas or anti-Jewish slurs). But this assessment is complicated by the fact that some general Jewish organizations do post pro-Israel signage outside, host Israeli politicians, or put on pro-Israel events such that protests against them may be responding to specific political messaging. The audit often doesn’t give enough information to determine whether this is the case. For example, the data says that an October 10th West Hartford, Connecticut online meeting hosted by a “Jewish organization” was “Zoom bombed” by someone who put “Free Palestine” on their screen for attendees to see. If the event was something like an online minyan or Hebrew class, then showing up to display “Free Palestine” implies that those involved in normal Jewish life should answer for Israel’s behavior. But if it was an online presentation about how to support Israel, as organizations around the country have held, then the assumption that antisemitism motivated the disrupter is unfair. For these reasons, we placed some cases of anti-Zionist messaging directed at Jewish organizations in the “need more information” category if we needed more context about the organization in order to evaluate them.
In addition to the 498 inconclusive cases, we also identified 629 incidents—about 7% of the total—in which the only evidence provided that an act was antisemitic was the ADL’s use of the word itself, such as “antisemitic comments,” “antisemitic imagery,” or “antisemitic beliefs.” When asked about whether or not the ADL has more information on each incident, a spokesperson for the organization said that they do, but do not share it publicly. “We have additional data for these incidents and all of them go through a thorough process to examine credibility, involving law enforcement, ADL regional offices and research experts, as well as independent verification when possible. We sometimes do not include additional information in the public version of the dataset for clarity, concision, and confidentiality purposes,” the spokesperson said. Yet without public specifics, it can be hard for researchers like us to understand what actually happened in each incident.
Once we finished reclassifying all the data, the items that we found to not involve any antisemitism account for around 17% of all the incidents labeled “antisemitic” in the ADL’s report. If we also exclude the aforementioned cases of seemingly random imagery—without a clear target, and limited in impact—the total number of antisemitic incidents logged by the ADL decreases by 30%, even if we give the organization the benefit of the doubt and include the two kinds of incidents about which we needed more information. If we omit those inconclusive cases, we find that only about 56% of the cases in the audit demonstrate clear-cut, unambiguous, targeted antisemitism.
AFTER COMBING THROUGH THE ADL’S DATA, it became clear to us that this reassessment couldn’t provide a clear picture of American antisemitism in 2023, because the audit does not sufficiently differentiate by degree and often does not share enough information to determine whether incidents are clearly antisemitic. Still, our analysis yielded some helpful takeaways.