What is AI?
AI stands for “artificial intelligence” and simply means any computer system designed to perform tasks that typically require human intelligence. AI includes a range of applications, from the natural language processing used in large-language models (LLMs) that lets Google answer questions like “Who is Captain America’s best friend?”, to the machine learning programs used to predict protein structures, to the decision trees that determine opponent actions in video games. AI can be generative (creates new content), assistive (makes someone’s job easier), or agentic (makes decisions and takes actions independently). When people casually refer to “AI,” they may mean any of these things. When we talk about the use of AI to create book indexes, we’re referring to LLM-based generative AI with extensive natural language processing capabilities.
How do generative AI and LLMs differ from other types of AI?
Generative AI refers to AI that creates new content derived from the content it has ingested and been trained on. Generative AI that creates textual content is based on large language models, or LLMs, and works by predicting which words or phrases would be most likely to satisfy the user-provided input or prompt. These predictions are based on automated statistical analysis of huge amounts of textual content, such as books, articles, blogs, and websites. The resulting output mimics human speech but does not reflect any actual understanding on the part of the AI.
Can generative AI produce a book index for me?
Generative AI applications such as ChatGPT and Claude can be prompted to generate a plausible index—that is, a document that superficially resembles an index—but cannot at this time generate an adequate index. Our initial evaluations of AI-based indexing software suggest it shares these limitations; we are in the process of conducting more in-depth evaluations.
Does AI make good book indexes?
Generative AI does not create adequate indexes, let alone good ones. Indexes produced with generative AI fail to effectively guide readers to subtopics and related topics, fail to include all indexable material and simultaneously clutter the index with unnecessary entries, fail a standard copyeditor’s test for accuracy, and contain many other issues that would need to be remediated before publication.
What kind of quality problems do AI-generated book indexes show?
Indexes produced by generative AI show issues with navigation (guiding readers to subtopics and related topics via appropriate use of cross-references), with both under- and over-indexing (they omit terms that should be included and include terms that should be omitted), and with accuracy (they fail a standard copyeditor’s test for page locator accuracy). In our tests, 70% or more of AI-generated entries would require human remediation before publication. In most cases, this is likely to take longer (and be more costly) than indexing the book from scratch.
Why are inappropriate cross-references a problem?
Cross-references guide the reader to related information under different headings, ensuring that all relevant information can be easily found while maintaining a clear organizational structure so that the reader does not become overwhelmed with irrelevant information. They also provide a way to direct readers from common terms they may look up to the preferred term(s) used by the author.
AI-generated indexes make very poor use of cross-references, sometimes using far too few and other times using far too many. When they do use cross-references, they tend to use them inaccurately. For example, they include both cross-references that lead to nonexistent entries, which confuses the reader and wastes their time, and unnecessary cross-references where the page numbers should simply be posted under both headings or where the pages at the target entry should be added to the original entry.
Why is under-indexing a problem?
An index needs to guide the reader to all relevant information in the text. Under-indexing means that some material that should be included in the index is left out. A reader relying on the index to find information might mistakenly conclude that omitted information is not present in the book. Depending on the specific AI product used, AI-generated indexes may under-index names, titles of works, or other terms and may inappropriately omit chunks of the book from the index. To date, no AI product we have tested effectively indexes all of the above.
Why is over-indexing a problem?
Indexes should be concise, quickly guiding the reader to relevant information. Over-indexing occurs when unnecessary entries are included in the index; this clutters the index, wastes the reader’s time, and increases printing costs. AI-generated indexes may contain excessive redundant subheadings and entries for passing mentions (insignificant topics that should not be in the index).
Why is inaccuracy in a book index a problem?
If the page numbers in the index are wrong, readers will not find relevant information when they go to the indicated page. This wastes their time and may cause them to lose trust in the index, lose trust in the author (if they assume the author created the index), or even lose trust in the book as a whole. In our tests, AI-generated indexes contained enough page-number errors that every entry would have to be manually checked before the index could be considered reliable and accurate.
I’m an indexer. Can I speed up my work by using an index produced by generative AI as a draft?
In tests of generative AI for producing indexes, 70–100% of the entries showed various types of mistakes or best-practice violations that would require remediation. More importantly, however, AI-generated indexes both omit material that should be indexed and include material that should not be indexed. Remediating these issues would require going through the text, checking each entry, and determining what indexable material must be added to the index and what non-indexable material needs to be deleted from the index. This is an extremely time-consuming task, and since it’s essentially the same process an indexer would follow if they were indexing from scratch (but with additional flipping back and forth between index and text), no time will be saved.
I’m an author. Can I make a book index using generative AI and then save time and money by having my indexer work from that?
Remediating the issues with AI-generated indexes requires going through the text, checking each entry, and determining what indexable material must be added to the index. This is extremely time-consuming and is essentially the same process a professional indexer would follow if they were indexing from scratch, with additional flipping back and forth between index and text. Professional indexers generally find it quicker to index from scratch than to edit even a good human-created index in this fashion. Thus, editing an AI-generated index is likely to take longer and cost more than creating a professional index from scratch.
What value do human indexers bring to the indexing process?
Human indexers can understand the nuances, connections, and implied or assumed information in the material being indexed, allowing them to anticipate reader needs and reflect authorial intention. In the process of working through a text, indexers develop a sense of its structure, which they then ensure is reflected in the index. They can imagine different possible approaches readers may take and create index entries that support a variety of reader approaches to finding the desired information.
Are there any ways generative AI can be useful in book indexing?
In some recent tests, Claude generated entries for all relevant names in the text, as well as some that were only mentioned in passing and should have been omitted from the index. If this holds true for book-length material—and an informal recent test suggests it does not—then Claude could theoretically produce a list of names that a professional indexer could then check, remediate, and add to an index. However, without checking the entire book, the indexer would not know if any relevant names had been omitted. Further research is needed to determine whether Claude can pick up all names from a book-length work, including non-Westernized names.
What are the ethical concerns around using generative AI in book indexing?
The ethical concerns around generative AI in indexing primarily relate to client privacy and consent. Indexers should disclose their use of generative AI and should never upload client material to an AI product without express client permission. In addition, indexers should consider broader ethical concerns surrounding AI such as environmental impact, systemic bias, and intellectual property theft.
Can AI-based indexing software make a good book index? Should I use AI-based indexing software to index my book?
Indexes created with AI-based indexing software show issues similar to indexes generated via direct prompting of an AI, including issues with index structure, unhelpful and inappropriate cross-references, and under- and over-indexing. They also show additional issues such as unhelpfully long page ranges and failure to break down entries with appropriate subheadings. For reviews of specific AI-based indexing software, see the American Society for Indexing website’s section on AI and indexing.
Why aren’t LLM-based AIs good at generating a book index?
We aren’t entirely sure, but we have some theories. First, LLMs ingest vast amounts of content without any supervision, and treat all content as of equal value—that is, no human tells them “This is a good idea, this is a bad idea; this is a good index; this is a bad index.” Second, LLM-based AIs operate not by exercising a skill but by predicting likely responses based on similar text that they’ve ingested. When a professional indexer works on a book, they bring all their skills to bear in order to focus on that book specifically and in isolation. Third, LLM-based AIs are essentially giant statistical engines; they work by reducing text to tokens or mathematical representations. Indexing requires a deep understanding of language as language—how readers interact with a text, subtle nuances of meaning, and unspoken or implied information. Finally, LLMs are trained on vast amounts of data, but indexing is about depth, not breadth; how other books were indexed has no bearing on how this book should be indexed.
Where can I learn more about AI and book indexing?
The AI Committee of the American Society for Indexing maintains a section on its website on AI and indexing, which includes research, reviews, and links to related resources.