ChatGPT Cannot Provide Useful Legal Advice
Most people will have seen the headline: “ChatGPT Passes US Law School Exam” and concluded that artificial intelligence (“AI”), and ChatGPT in particular, would soon be doing lawyers’ work. In fact, ChatGPT barely passed the law school exam, which (as such exams go) did not ask students to cite legal precedent. Curious about how ChatGPT would perform in answering real-world legal questions, I engaged ChatGPT on two topics: trademark and copyright, which are relevant to our practice. The results were not only terrible, but shockingly so for anyone who believed in the power of current AI.
ChatGPT Flubs the Trademark Test.
In my first inquiry, I asked, "Explain the rule of foreign equivalents used by the United States Patent and Trademark Office in reviewing trademark applications."
ChatGPT quickly responded:
" The United States Patent and Trademark Office (USPTO) uses the "foreign equivalents" rule when reviewing trademark applications. This rule states that a trademark that is similar to a foreign word or phrase may be considered confusingly similar to a mark that includes the English translation of that foreign word or phrase. In other words, if a proposed trademark is the same or similar to a foreign word or phrase that translates to the same or similar meaning as the proposed trademark, the USPTO may reject the application on the grounds that it is likely to cause confusion among consumers. This rule is intended to prevent confusion among consumers and to protect existing trademark holders from infringement. "
The first thing I noticed (besides the convoluted explanation) was that ChatGPT missed an essential element of the doctrine: that it is applied to marks that the ordinary American purchaser would likely stop and translate into its English equivalent. (The ordinary American purchaser includes all American purchasers, including those proficient in a non-English language who would normally be expected to translate words into English.) Our firm has cited this element of the doctrine to overcome a foreign-equivalents objection cited in an Office Action where the language in question was spoken by a small percentage of Americans.
Following up on the foreign-equivalents inquiry, I asked ChatGPT, "What are the leading cases in the Trademark Trial and Appeal Board concerning the rule of foreign equivalents?"
ChatGPT responded:
" The United States Patent and Trademark Office's Trademark Trial and Appeal Board (TTAB) has issued several precedential opinions concerning the "foreign equivalents" rule. Some leading cases include:
In re American Fertility Society, 230 USPQ 481 (TTAB 1986) - The TTAB found that the mark "Fertilite" was likely to be confused with the mark "Fertility" because the foreign equivalent of "Fertilite" is "Fertility."
In re Societe Comptoir de L'Industrie Cotonniere Etablissements Boussac, 229 USPQ 726 (TTAB 1986) - The TTAB found that the mark "Boussac" was likely to be confused with the mark "Boussac - The Fabric of the World" because the foreign equivalent of "Boussac" is "Boussac - The Fabric of the World."
In re Societe Generale des Eaux Minérales de Vittel, S.A., 222 USPQ 1235 (TTAB 1984) - The TTAB found that the mark "Vittel" was likely to be confused with the mark "Vital" because the foreign equivalent of "Vittel" is "Vital."
In re Société des Produits Nestlé S.A., 96 USPQ2d 1570 (TTAB 2010) - The TTAB found that the mark "Nestle" was likely to be confused with the mark "Nest" because the foreign equivalent of "Nestle" is "Nest."
In re Red Bull GmbH, 59 USPQ2d 1213 (TTAB 2001) - The TTAB found that the mark "Red Bull" was likely to be confused with the mark "Red Bull Energy Drink" because the foreign equivalent of "Red Bull" is "Red Bull Energy Drink."
These are some of the leading cases decided by the TTAB, it is important to consider that TTAB decisions are not binding on US District Court, but they may be considered persuasive authority. "
Looks legit, right? The problem is that these 5 cases are nowhere to be found.
On the Westlaw database, I searched for the decisions in All Federal, the TTAB section (decisions of the Trademark Trial and Appeal Board), and the leading treatise on trademark law by McCarthy. In each of these resources, I searched by name, by citation number, by individual marks (Fertilite, Fertility, Boussac, Vittel, Nestle, and Red Bull Energy Drink). No such case was found.
I did the same on Google Scholar – nothing.
Then I went to the Trademark Manual of Examining Procedure (latest edition July 2022) at the United States Patent and Trademark Office. The TMEP is used by Trademark Examiners and attorneys to refer to leading cases affecting the registrability of trademarks. None of the cases listed by ChatGPT were found. Unless I missed something, ChatGPT tried to pull a fast one, inventing its own cases. Even so, several of the case explanations above make no sense. And “vittel” is not French for vital. It’s the name of a French town. The leading case for Vittel is about geographic significance, not foreign equivalence – a completely different issue.
I tried again, but ChatGPT doubled down with fake news. I asked: “What are the leading cases in the United States Trademark Trial and Appeal Board regarding the doctrine of foreign equivalents?
ChatGPT responded with a new list of allegedly “leading cases in the United States Trial and Appeal Board (TTAB) regarding the doctrine of foreign equivalents,” but this time it provided no case summaries. Of the five cases cited, three were fictions created by ChatGPT, and the two real cases have nothing to do with the doctrine of foreign equivalents. My comments follow each of the citations:
In re Merrill Lynch, Pierce, Fenner & Smith, Inc., 828 F.2d 1567 (Fed. Cir. 1897)
Although this is real case regarding trademarks, (a) it was in federal court, not the TTAB, and (b) in this appeal from a TTAB decision denying registration of the mark, CASH MANAGEMENT ACCOUNT, there is no mention of (or issue regarding) foreign equivalents.
In re E.I. du Pont de Nemours & Co., 476 F.2d 1357 (C.C.P.A. 1973)
This is a landmark case that established what are known as the “duPont factors” for determining consumer likelihood of confusion, but it makes no mention of the foreign equivalents.
In re Niday, 375 F.2d 1000 (C.C.P.A. 1967)
This is another ChatGPT invention. In reality, 375 F.2d 1000 is a decision in Smith v. Taylor, a Florida federal court dispute regarding labor and employment, and a search for “In re Niday” only turns up a 2013 bankruptcy court case. In a TTAB search, “Niday” turns up nothing.
In re La Coste, 30 USPQ2d 1053 (TTAB 1991)
Finding that this was yet another fictional creation, I searched for Lacoste and found 18 TTAB decisions, but Lacoste was mentioned only for “the dominant role fame plays in the likelihood of confusion analysis,” not foreign equivalents. In one of the latter cases, Syndicat des Proprietaires Viticulteurs de Chateauneuf-Du-Pape v. Pasquier Desvignes, 107 U.S.P.Q.2d 1930 (TTAB, June 14, 2013), the TTAB applied the doctrine of foreign equivalents without explanation or citation to the doctrine, merely noting that translations into English of the two marks, CHATEAUNEUF-DU-PAPE and CHEMIN DES PAPES, would not cause confusion and that, moreover, “purchasers may not know French, in which case they will not translate either mark. In that case, there is even less chance of confusion.” ChatGPT gets no credit for my jump down the rabbit-hole of case decisions.
In re Honda Giken Kogyo Kabushiki Kaisha, 40 USPQ2d 1020 (TTAB 1996)
This is yet another fictitious decision. A search of TTAB decisions in Westlaw found 88 cases with the words “Honda Giken Kogyo” Two of them, Ricardo Media Inc. v. Inventive Software, LLC (TTAB 2019), and Semiramis v. MDJ Nemry Corp. D1B1A Semiramis USA (TTAB 2021) (only one of them precedential) applied the doctrine of foreign equivalents. In both, there was a citation to Kohler Co. v. Honda Giken Kogyo K.K., 125 USPQ2d 1468, 1478 (TTAB 2017) – not for foreign equivalents, but for an issue regarding TTAB trial procedure.
The leading precedent regarding the doctrine of foreign equivalents is actually an appeal from the TTAB (affirming the TTAB’s decision), Palm Bay Imports, Inc. v. Veuve Clicquot Ponsardin Maison Fondee en 1772, 396 F.3d 1369, 73 U.S.P.Q.2d 1689 (Fed. Circuit 2005). But there are 21 precedential cases in the TTAB regarding the doctrine, each of which is helpful to the practitioner faced with a refusal by the USPTO to register a client’s trademark based on the doctrine.
ChatGPT Fails the Copyright Test.
My next inquiry concerned copyright. I asked ChatGPT, “When is publication of portions of unpublished works protected under the "fair use" doctrine?”
ChatGPT responded:
" Publication of portions of unpublished works is protected under the "fair use" doctrine if it is used for a limited and transformative purpose such as criticism, commentary, news reporting, teaching, scholarship, or research. The specific circumstances surrounding the use, including the purpose, the nature of the copyrighted work, the amount used, and the effect on the potential market for the work, will be considered in determining whether the use is fair."
I first noted that ChatGPT did not have a solid grasp of the issue. With respect to unpublished works, the burden of demonstrating fair use is heavier than where published works are concerned. Following up, I asked, “What cases support the proposition that the publication of portions of unpublished works is protected under the ‘fair use' doctrine?”
ChatGPT responded:
" There are several cases that have supported the proposition that the publication of portions of unpublished works is protected under the "fair use" doctrine. Some examples include:
Harper & Row Publishers, Inc. v. Nation Enterprises (1985), where the U.S. Supreme Court found that the publication of quotes from an unpublished memoir by President Gerald Ford was protected under fair use because it was used for the purpose of commentary and criticism, and the amount used was relatively small.
New York Times Co. v. Tasini (2001), where the U.S. Supreme Court found that the republication of freelance writers' articles in electronic databases without their permission was not protected under fair use because it was not transformative and it competed with the market for the original works.
Bill Graham Archives v. Dorling Kindersley Ltd. (2007) where the court found that the use of images of concert posters in a book was protected under fair use because it was used for the purpose of historical commentary, was highly transformative and the use did not harm the market for the original works.
These cases are not exhaustive and the fair use doctrine is a flexible principle, the determination of whether a use of copyrighted material is fair use is a case-by-case analysis based on the specific circumstances of the use. "
The above response from ChatGPT may sound scholarly, but ChatGPT could not have gotten things more wrong. In Harper & Row Publishers v. Nation Enterprises, the Supreme Court found that The Nation magazine’s publication of excepts from Gerald Ford’s unpublished memoirs was not fair use. Perhaps if The Nation had not taken what the Court considered to be the "heart" of the memoirs, the Court might have ruled in The Nation’s favor, but as it was, The Nation was adjudged guilty of copyright infringement.
The second case, NYT v. Tasini, did not involve the “fair use” doctrine. There, the Court noted that very fact: "Nor do [the Defendants] assert that the copies in the Databases represent “fair use” of the Authors' Articles."
The third case, also from the Second Circuit, concerned re-publication of Grateful Dead concert posters. The case summary above also misses the confluence of factors in Dorling Kindersley, among which the small reproduction of the posters was crucial. Oddly, ChatGPT uses the phrase "highly transformative" above, but the court used that particular phrase only once - in its summary of Kelly v. Arriba Soft Corp. (concerning thumbnail images). In addition, Bill Graham Archives was decided on May 9, 2006, not 2007 as ChatGPT stated.
Conclusion.
ChatGPT not only provided inaccurate answers, but it failed even as a tool that could make the job of lawyering easier or less time-consuming. Anyone who relies on ChatGPT for answers to legal questions has a fool for a lawyer, at least for now.