Magic History Research on the Internet: Digital Newspapers
By Gary Hunt
Researching magic history has changed a lot in the last few years with the growing number of searchable newspaper databases. When working on Gus Rich I spent almost 5 years going through microfilms of every contemporary Salem, North Carolina newspaper. If they had been in a searchable database I could have been finished in a matter of days! The use of these databases will change the way magic history is researched and even how it is currently understood. I have been able to track Fergus Greenwood’s (Fasola) early career though fleeting mention in articles, small reviews, tour routes and small advertisements in various newspapers and trade journals. How else could I find out that he had been convicted of stealing from a lodging house when he just started out, the succession of names he used, or how his act changed and grew over the years.
So where does one start? It really depends on what you are looking for, how much you want to spend and how much time you have. The good news is that the number of newspapers pages being digitized is growing by the millions each year. The bad news is that this is being done by a wide range of organizations, from individuals to multi-national corporations.
Before we get much farther, one has to understand how digitized newspapers are created and searched. In most cases existing microfilm is put through a high speed scanner and an image is taken of each page of the newspaper. The images are then analyzed by Optical Character Recognition (OCR) software, which reads each image and converts it into text. The text is linked back to the image and everything is put into a searchable database. So in a perfect world the software has converted every word in the image into perfect text. However, this is how it really looks:
You can see that the OCR is not 100% accurate and that the microfilm can be of rather poor quality. This will vary with the newspaper databases depending on the quality of the microfilm, the equipment and software used and if any quality control procedures were followed. This is actually a very good OCR conversion. You have to keep this in mind as you develop your search strategy.
Most of the current databases only cover newspapers published before the mid-1920s. This cutoff date is related to copyright issues. There are exceptions to this and more 20th century runs are being digitized. Many of the newspapers are from small towns, which is wonderful if you are looking for “tall grass magicians”. Unfortunately numerous major newspapers, such as the Boston Globe or the Los Angeles Times, are in pay for use databases, if they are in any at all.
Newspaper databases breakdown into two types. Those that are national in scope and those that are state/regional/local. They can be further divided into fee for service and free. In many cases the databases that cover state, regional and local newspapers are free.
Fee based English language newspaper databases
There are a number of major companies that charge a fee to access their collections of digitized newspapers. Some of these collections can be directly accessed from a home computer, while others can only be searched at a library which has a license. These include:
Newspaper Archive (Heritage Microfilm)
Comments: Heritage Microfilm claims that this is the largest on-line historic newspaper database and is growing at the rate of 2.5 million pages a month. The collection covers English language newspapers from 1752 onwards and is mainly focused on US publications. It contains a lot of small city papers, great for finding those obscure performers. Many libraries subscribe to this database. It is also the newspaper database used by Ancestory.com.
Comments: This is the public access site to the complete digital historical newspaper collection from Readex, a division of NewsBank. It covers over 5,800 newspapers from 1690 to today, with the majority printed before the 1920/30’s. Many of the newspapers are from the upper Atlantic coast and New England states, though it does have national coverage.
The British Newspaper Archive
Cost: The collection can be searched for free. However, depending on the package purchased there is a cost of £0.01 to £0.07 per page to view and print the results. An unlimited plan is available for £79.95 per year. The collection is also available to members of the genealogical site www.findmypast.com .The fee is £109.95 per year, but shorter term memberships are available.
Comments: This replaces the “19th Century British Library Newspapers” collection. The collection will eventually contain over 40 million newspaper pages covering the years 1700 to 1999. It currently has 210 newspapers and over 6 million pages. The best news is that it includes The ERA. This publication is the paper of record for entertainment in England and would be comparable to the New York Clipper, Variety and the Billboard combined. This is the Holy Gail for 19th Century magic research. Be warned, once you begin to search The ERA you will not be sleeping for days.
Proquest Historical Newspapers
Cost: only available to libraries and institutions
Comments: Proquest is one of the big boys on the block. They provide their databases only to libraries and institutions. Subscription fees are very expensive and this is one reason that only big organizations can afford them. Also, they are very strict on how their license holders provide remote access to the databases. Several years ago Proquest stopped licensing its databases to membership organizations that allowed remote access. This was the reason I had been a long time member of the Society for American Baseball Research. Proquest also does not provide direct public access or subscription services to individuals.
Proquest has digitized some of the major US newspapers including near complete runs of the: Atlanta Constitution, Los Angeles Times, New York Times, Pittsburgh Courier, Wall Street Journal, Philadelphia Tribune, Boston Globe, Chicago Tribune, San Francisco Chronicle and the Washington Post. They have also digitized The Guardian & The Observer, Irish Times and The Scotsman. Most major universities and some state/local libraries have access to some or all of the newspapers. The public can search the newspaper archives for free on the respective newspaper’s website, but a high fee is charged to see and print the image. The New York Times is one exception, articles before 1922 and from 1987- today are free to view and print/save the articles. This is a key source for theatrical information.
Gale (Cengage Learning)
Cost: Available to libraries and institutions, with some limited direct public access
Comments: This is the other big boy and some of their more interesting databases have been developed in cooperation with the British Library. Gale only makes their databases available to libraries and institutions. However, what makes them different from Proquest is that they are more flexible in allowing libraries to offer remote access to the collections they subscribe too. Some of the key databases include:
- 19th Century U.S. Newspapers: This is a static collection of newspapers selected for their importance by an editorial board consisting of American historians. The collection consists of about 1.7 million pages and is national in scope
- 17th & 18th Century Burney Collection Newspapers: This collection, housed in the British Library, was accumulated by Rev. Charles Burney (1857-1817) and includes more than 700 bound volumes and 1,200 pamphlets, proclamations, newsbooks and newspapers. Here you can find some of the earliest accounts of English magicians.
- 19th Century British Library Newspapers: This is a static collection of 48 British newspapers (2.2 million pages) from the 1800’s. It does include the The ERA. This collection has been combined with the new and expanding “British Newspaper Archive” discussed above.
- The Times Digital Archive, 1785-1985: The most important paper in England. What more can be said.
Again, many major universities and some state/local libraries subscribe to one or all of these databases.
Cost: $59.95 per year
Comments: This is a small collection of early American newspapers and historical records. Examples include The South Carolina Gazette (1734-1780) and the Pennsylvania Gazette (1728-1800).
Irish Newspaper Archives
Cost: Prices range from 10 Euros per day to 350 Euros per year
Comments: This database currently contains full runs of 21 major Irish newspapers dating from the 1700’s until today. A great, yet very expensive resource. It is best to do a free search to see what turns up and then buy a one day pass, which allows unlimited viewing and printing/saving. Also recommend brewing a very strong pot of coffee because it could be a long night.
There are several ways that these newspaper databases can be accessed. First check with your local library or university. In many cases they may have subscriptions to a few collections that can be accessed for free at the facility or even remotely. State libraries may also have licenses that will allow for off-site searching. If none of these options are available, then be prepared to spend a few bucks.
Here are some suggestions for accessing many of these collections:
- If you belong to Ansestory.com then you already have access to NewspaperArchivce.com . Joining Ancestory.com costs $155 per year and up depending on the resources you need to access.
- Join Godfrey Library. This is a genealogical library located in Middletown, CT that provides its members with on-line access to a wide variety of databases and other resources. For $80 per year you can access a wide variety of historic and genealogical resources and newspaper databases, including: 19th Century U.S. Newspapers (Gale), Early American Newspapers (Newsbank), 19th Century Newspapers (Accessible Archives), and London Times Digital Archives (Gale) and newspaperarchives.com. The Godfrey Library is a great source of on-line historical and genealogical resources from across the world and a real bargain.
- Join GenealogyBank to access all of the NewsBank databases. The cost is $69.95 per year and well worth it.
- Marry a University professor and remotely access their library account. Worked for me.
Free national newspaper databases
A number of national institutions and private organizations have started to digitize newspapers and made the databases freely available. Some of the largest collections are listed below.
Chronicling America, Early American Newspapers (Library of Congress)
Comments: Chronicling America is sponsored jointly by the National Endowment for the Humanities and the Library of Congress as part of the National Digital Newspaper Program. The database covers 1836 to 1922 and currently includes selected newspapers from 28 states. As one would suspect this is a very high quality product. See your tax dollars doing some great work
Small Town Papers
Comments: As the title suggests, this is a database of over 250 small town newspapers. Most are from the last few decades, but some go back into the 1800’s. It is indexed by Google and can be searched either at the site or from Google’s news achieve.
Papers Past (National Library of New Zealand)
Comments: This collection contains more than one million pages of digitized New Zealand newspapers and periodicals. The collection covers the years 1839 to 1945 and includes 72 publications from all regions of New Zealand.
Australian Newspaper Digitisation Program (National Library of Australia)
Comments: The Australian Newspapers Digitisation Program has digitized approximately 3 million pages from over 50 newspapers. This covers a range of titles from every state and territory, from the earliest newspaper published in Australia in 1803 through to the mid 1950’s.
Google News Archive (Google)
Comments: Google has gotten into the digital newspaper area in a big way. They purchased Papers of Record, a database of historic newspapers rich in Canadian and Spanish language publications. Google is also working with partners to digitize additional newspaper runs. They have also indexed some of the newspaper databases on the internet, such as Small Town Papers and Proquest. Google is doing what it has done with books and thorough its partnerships with libraries, historical societies and publishers there soon will be a significant increase in the availability of digital historical newspapers. Google could end up being the biggest and badest boy on the block.
Free local, regional and state databases
Now we get into what the internet is all about. Organizations and individuals putting up information anyway they can. Hundreds of newspapers have been digitized and put into searchable databases by libraries, newspaper publishers and just interested folk. Who would of thought that when I started to research Fredrick Bancroft all three newspapers from his hometown of Winona, Minnesota, would be on line. What a goldmine of information.
Keeping up with what is out there can be a challenge. The most complete list of US newspapers is the Research Guides: Historical Newspapers Online maintained by University of Pennsylvania Library. A source for international digitized newspapers is the International Coalition of Newspapers (ICON). Also check the List of Online Newspaper Archives Wiki .
One of the top sites for doing research is the oddly named Old Fulton NY Post Card Website. Tom Tryniski, the site administer, has undertaken a herculean task of scanning the microfilm of many New York newspapers. While this in itself would be a valuable asset to the magic historian, the key to the site are the files of the New York Dramatic Mirror (1831-1922) New York Clipper (1853-1924), Billboard (1894-1922) and Variety (1906-1922). These publications are the Holly Grail of historic theatrical research for the mid 1800’s to the early 1900’s.
This is a godsend for anyone who has had to set in the dark for hours on end going through reel after reel of microfilm. Though, anyone who has seen the quality of some of the Clipper and early Billboard microfilms will know the greatest care was not taken in their production. So the images can be hard to read and the search of the digital copy may not find all references.
The site itself can seem a little daunting, so do read the FAQ page. Using the “Boolean search” function can be a powerful way to get very specific search results. Though simple searches using multiple terms seems to get one close. Be warned, you will find yourself chasing down other bits of fascinating information and soon forget what you were looking for in the first place. This is a self-financed effort so do donate a few dollars. This is one site that we do not want to lose.
A simple search in anyone of these databases can turn up hundreds if not thousands of articles and many will be irrelevant. So how does one sift the wheat from the chaff? Here are a few suggestions that I use. However, sometimes it is best to just get a cup of coffee and dig through them all. You never know what you may find.
- Become familiar with advanced search techniques using wildcard characters and Boolean logic. This is not as complicated as it seems. Most search engines provide a “fill in the box” type advance search based on Boolean logic. Wildcard characters are special symbols that represents one or more characters. For example, searching for “run*” will turn up run, running, runs, runners, etc.
- Try to use specific search terms. Use multiple words or phrases and then reduce the number if you do not get the results you need.
- Try to refine searches by dates. This could be over a time period, a month or a specific date.
- If you get a hit look through the rest of the papers from that week. Remember that acts usually played from Monday through Saturday and traveled to the next town on Sunday. So check the Sunday (or Saturday) newspaper for publicity articles or advertisements. You sometimes find a review from another town mentioned. The review of the act is usually in the Tuesday paper.
- Try searching the names of partners in the act, assistants, title of the act, type of act, or feature effect. Many times when the text is OCRed only parts of the articles are recognized. So any bit of information helps. For example when I searched for Vano & Anvo escape artists. I searched Vano Anvo; Vano; Anvo; Vano escape, Vano handcuffs, etc. So it can get a little overwhelming depending on the number of variables. However, I am always amazed what can turn up.
- One approach I have taken is to use the theatrical publications to track a magician’s tour and then use local newspapers to find more specifics.
- Use research librarians in the cities the magician you are researching has visited. Librarians are a greatly overlooked resource. Only a fraction of historical newspapers are on-line. Find the local library and send them a nice e-mail. I always include some history of the performer and attach a picture or two. Make sure you give them the dates you want looked up and do not ask for too much. You will get some amazing feedback, such as I have not had such a fun request in years. You may be asked to pay a small free for copies and it is always well worth it.
- Join as many databases as you can afford. Some, such as the newspapers indexed by Proquest, have a daily or per view rate. So you can pay for a day or a number of views and do a focused search. Many of the pay sites allow free searching of the databases, but one has to pay to view the document. So you can use the search results to track down the issues on microfilm.
- Many of the databases are constantly being updated. So keep searching, especially when a research project lasts years. As mine have a tendency to do.
- Don’t overlook Google, it is a great place to start.
Copyright December, 2012