LibGuides: Researching and Writing a Paper: Reliability of Sources

Techniques for Evaluating Resources

No matter how good the database you search in is, or how reliable an information website has been in the past, you need to evaluate the sources you want to use for credibility and bias before you use them*. (You may also want to spot fake news as you browse the Internet or other media - not all fake news is online).

This page discusses eight different tools for evaluating sources (there are so many different tools because evaluating the reliability or quality of information is an important topic, because there are many ways to look at the topic, and every librarian wants to help you succeed). Look through these approaches and use the approaches or combinations of approaches that work for you. The tools are:

5Ws (and an H)
A.S.P.E.C.T.
ABCD
Authority
Evaluating Research Articles
Lateral Reading ("what do other sources say?")
SIFT
The CRAAP Test

We also have a variety of videos about evaluating sources available for your learning and entertainment: Click Here!

* Note: a biased source - and technically most sources are biased - can be a useful source as long as you understand what the bias or biases are. A source that is trying to be reliable will often identify some or all of its biases. (Every person has a limited perspective on the events they observe or participate in, and most of the time their perceptions are influenced by assumptions they may not be aware of. So, even when you have some really solid reasons to trust a source as 100% reliable and accurate, be alert for 'what is not mentioned' and for what biases there might be [this is particularly tricky when you share those biases, and why reviewing your research/paper/presentation with someone else is always a good idea]).

5Ws (and an H)

The 5Ws and an H.

What?	What is the document (opinion, news article, review, report, research study, popular article, scholarly article, blog post, peer-reviewed article, statistical analysis, corporate document, government document, etc.)? Is it a primary (created by a participant or observer of the events), secondary (created by someone who has read several primary documents), or tertiary document (created by someone who has read several secondary documents)? What tone does the author use - conversational, factual, academic, etc.? Did the document go through a vetting, editing, or peer review process?
When?	When did the research or event the document discusses happen? When was the information published and/or updated? Do you need only the most updated information, or is your topic more historically oriented?
Where?	Where was the was the document published? What kind of publication is it (Academic Journal, Trade Journal, Newspaper, Website, Magazine, Encyclopedia, Book, Corporate or Commercial site, etc.)? Is the publisher a known and respected source of information? (If the source is a web site check the domain name for clues (.edu, .org, .com, .mil, .net) to determine what type of page it might be; is there an "about" or "what is" link from either the information page or the "home page" that describes the purpose of the webpages? Are they trying to sell something?
Who?	Who is the author? What credentials does the author have that suggest they are knowledgeable? If an author is not named, who is the editor or sponsor? If the source is a web site, is there a link to a "home page" that says who is sponsoring the website? Can you detect any conflict of interest or potential bias in this author?
Why?	Why did the author write this document (presenting facts, making an argument, original research, etc.)? Who is the intended audience? What is the author's purpose - inform, persuade, entertain, share a point of view? Does this purpose seem honest and trustworthy? Was the author paid for their opinion by a third-party that might be biased? Does who wrote the information and where it was published indicate purpose? Does that purpose affect the reliability of the document?
How?	How did the author reach their conclusions? How did the author gather data to create the document? Did the author: gather data or information from credible outside sources; incorporate in-text citations and a list of references or works cited; present supporting pieces of data, sources, citations, quotes, personal experience, a reliable methodology? If there is no "works cited" page or "bibliography", are there any internal references to credible sources? Do these sources supplement the information in the document? Do the links/citations work? Did the production of this information go through a vetting, editing, or peer review process?

Back to the top of the page

A.S.P.E.C.T.

A: Authority	Is there an author’s name? Can you locate the author’s credentials? Can you find evidence of author expertise in the subject? Have you located similar works by this author? Do you have personal recommendations for this author? Do you know the publisher’s credentials and reputation? Are there similar works from this publisher?
S: Sources	Is information presented as fact? If yes… Does the author provide documentation? (Bibliography, footnotes, links, etc.) If documentation/sources are included, are they from credible sources?
P: Purpose	Was this source written to inform and educate? Does the source argue a perspective or specific opinion? Is the source intended to entertain or sell? Is the content aimed at a general audience, or is it written for readers with expertise in the subject? Is the source too basic, too technical, too advanced? Is the source just right for your research needs?
E: Evenness	Does the author recognize other points of view? Is the information presented objective? If the source is biased, does the author acknowledge the bias?
C: Coverage	Is the information new? Does it support what you have found in other sources? Does it contradict what you have found in other sources? Is the source comprehensive or inclusive enough for your needs? Does this source provide information that is relevant to your needs?
T: Timeliness	When was the source published? Is the date appropriate for your topic?

Back to the top of the page

ABCD

Authority
- Who are the authors or creators?
- What are their credentials? Can you find something out about them in another place?
- Who is the publisher or sponsor?
- Are they reputable?
- What is the publisher’s interest (if any) in this information?
- If it's from a website, does it have advertisements?
Bias
- Is this fact or opinion?
- Is it biased? Can you still use the information, even if you know there is bias?
- Is the the site trying to sell you something, convert you to something, or make you vote for someone?
Content
- What kind of information is included in the resource?
- Is content of the resource primarily opinion? Is is balanced?
- Is it provided for a hobbiest, for entertainment, or for a serious audience?
- Does the creator provide references or sources for data or quotations?
Date
- How recent is the information?
- Is it current enough for your topic?
- If the information is from a website, when was the site last updated?

Back to the top of the page

Authority

Authority
Information resources are a product of their creator's expertise and reliability, and are evaluated based on the information need and the context in which the information will be used. Authority is constructed: various communities often recognize different types of authority (knowledge, accuracy). Authority is contextual because you may need additional information to help determine the accuracy or comprehensiveness, and the sort of authority the source contains. (Writing a paper about 'urban myths' requires different sorts of authority than writing a paper disproving an urban myth.)

Using this concept means you have to identify the different types of authority that might be relevant, and why the author considers themselves reliable, as well as why their community considers them reliable. An author can be a person, journalist, scholar, organization, website, etc. Author is different from authority, authority is the quality that gives an author trustworthiness....and not all authors have the same trustworthiness.

Types of authority:

Subject expertise
Society standing
Life Experience

Trustworthiness depends on:

Where did the information come from?
Who was it created for?
How did they want you to use it?
What do you need it for?

Back to the top of the page

Evaluating research articles

Evaluating research articles: Evaluating evidence-based research articles in scholarly journals requires deep knowledge of the discipline, which you might not acquire until you are deeper into your education. These guiding questions can help you evaluate a research report, even if you are not an expert in the field. Questions include:

Why was the study undertaken? The aim of the research may be intended to generate income, lobby for policy changes, evaluate the impact of a program, or create a new theory. These variations in intent influence the research questions, the data collection, the analysis, and how the results are presented. To make best use of the findings for your purposes, you should keep the intent of the study in mind.
Who conducted the study? It is important to look at who conducted the research, and if the organization or individual in question has the expertise required for conducting research on the topic. Looking to see if the organization is interested in a specific research outcome is also a good practice. The research should be clear about how the different stages of the study were conducted to guarantee its objectivity.
Who funded the research? It is equally important to look at who sponsored or funded the study because this sometimes affects the objectivity or accuracy of the study. (If, for example, a soap-maker sponsors a study on the efficiency of different soaps, you should be critical of the results, particularly if their brand of soap is the best at cleaning.)
How was the data collected? In the social sciences, structured interviews and self-completion questionnaires are perhaps the two most common ways of collecting quantitative data. How the people in the study were recruited is essential for determining how representative the results are. (There are two main types of samples, probability and non-probability samples. A probability sample is one in which every individual in the population has the same chance of being included in the study. It is also a prerequisite for being able to generalize the findings to the population. Pretend you survey first-year students by asking student clubs to share the survey on their social media. This non-probability snowball sample is more likely to reach students active in the clubs, therefore the results will not be representative of, or generalizable to, all students.)
Is the sample size and response rate sufficient? The bigger the sample size the greater the chance that the results are accurate. After a sample size of around 1000, gains in accuracy become less significant. However, limited time and money often make such a large sample not practical. The similarity of the population also affects the desired sample size; a more diverse population requires a larger sample to sufficiently include the different parts of the population. The response rate is a complementary measure to the sample size, showing how many of the suitable individuals in the sample have provided a usable response. (In web surveys, response rates tend to be lower than in other types of surveys, and are therefore less accurate.)
Does the research make use of secondary data? Data can be collected for the purposes of the study or existing data gathered for a different study can be used. If existing data sets collected for another study are used, reflecting on how usable that data is for the newer study is important.
Does the research measure what it claims to measure? A commonly used word in statistics to describe the trustworthiness of research is ‘validity’. Validity refers to the extent to which an assumption or measurement is consistent with reality. Does it measure what it intends to measure? For example, a study investigates gender discrimination of faculty and looks at the number of cases of discrimination presented by female faculty. But, if the study does not look at the reason for these discrimination complaints (gender, ethnicity, religion, age, sexual orientation, etc.) it cannot be assumed that gender discrimination either increased or decreased.
Can the findings be generalized to my situation? There is often a tendency to generalize research findings. Two key standards have to be met to do this. First, results apply only to the population of the study. Second, data must be collected via a probability sample, i.e. everyone eligible to be in the study has the same chance of being included in the study. Too often papers do not discuss many of the aspects of the data collection and analysis. Transparently and clearly describing how the research was conducted is essential for the reader to understand the trustworthiness of the research paper in their hands.

Back to the top of the page

Lateral Reading

The Internet has democratized access to information, but the Internet has also been filled with a flood of misinformation, fake news, propaganda, and idiocy, presented as objective analysis. Since any single source is suspect, fact checkers read laterally. They leave a site in its tab after a quick look around and open up new browser tabs in order to judge the credibility of the original site.

Lateral reading is the process of verifying what you are reading while you are reading it. It allows you to read deeply and broadly while gaining a fuller understanding of an issue or topic and determining whether, or how much, to trust the content as presented.

Vertical reading occurs when the reader simply reads the article or site without going further, assuming that if it ‘looks reliable’ it is reliable. The reader may use some superficial evaluation strategies to determine if the site is credible, such as reading the ‘about’ page, looking at its URL extension (.edu, .org, .com, .gov, etc.), or assessing its advertising. A good start, but there is much more to look at:

Determine the author's reliability, intents, and biases, by searching for articles by other writers on the same topic (and also looking for other articles by that same author).
Understand the perspective of the site's analyses. (What are they assuming, what do they want you to assume?)
Determine whether the site has an editorial process or expert reputation supporting the reliability and accuracy of its content.

Use the strategies and ask the questions that professional fact-checkers use:

Go beyond the "about" page of the site you are reading.
Search for articles by other writers on the same topic.
Search for articles about the site/publication you are reading (and/or articles about the authors featured on the site).

Ask the following:

Who funds or sponsors the site where the original piece was published? (And who funds/sponsors the site you found the article at?)
What do other authoritative sources have to say about that site and that topic?
When you do a search on the topic of the original piece, are the initial results from fact-checking organizations? (If so, what do they say?)
Have questions been raised about other articles the author has written or that have appeared on that site?
Does what you are finding elsewhere contradict the original piece? (If there are contradictions, what is the reliability of those contradicting sites?)

Are reliable news outlets reporting on (or perhaps more important, not reporting on) what you are reading? (Does why reliable news outlets are or are-not reporting on the topic increase or decrease the reliability of the site you are assessing?)

Sometimes the 'good answer' to the above questions is a 'yes', sometimes a 'no', and sometimes 'it's complicated'. Reliable and unreliable sources are everywhere in the information we have access to - some sources are rarely reliable, but even the most 'consistently reliable sources' are sometimes unreliable (everyone has blind spots and biases, and everyone is able to make mistakes). There are no consistent rules for which questions must be answered which way. However, if you ask these questions and find out what the answers seem to be you will have a better understanding of how reliable or unreliable a particular source is.

Back to the top of the page

S.I.F.T. Method

SIFT (The Four Moves)

Use the SIFT method to separate fact from fake when reading websites and other media.

STOP Do you know the source of the information?
- What is its reputation?
- For deeper research, verify the information.

INVESTIGATE
- Know what you're reading.
- Where is it from? Biases, point of view?

FIND TRUSTED COVERAGE
- Understand the context of the information.
- Find the best source on the subject.

TRACE CLAIMS
- Trace claims, quotes or media back to its original context.
- Was the source you read/viewed an accurate depiction of the original?

More Information about the SIFT method, and a free 3-hour online course (five easy lessons) that will seriously improve your information evaluation skills!

Back to the top of the page

The CRAAP Test

Currency	Do you need current or historical information? (Most database searches allow the user to select the time range. If you are looking at a website, how recently has the website been updated?) ‘Current information’ is different for different topics. In healthcare and legal issues, three to five years is usually ‘current’. For history and literature ‘current’ is a much wider range of years.
Relevancy	Does the information relate to your topic or answer your question? Have you looked at several sources before determining this is a source you will use? What audience or level is the information written for? A general audience? Scholars? Students? Professionals? (Generally) begin with databases specific to your intended audience. This often means, for example, that for a medical audience you would select databases that have health and medicine journals – but there are often more relevant characteristics to your research question than just the medical perspective. Spending a little time figuring out what kinds of information are relevant to your question will help you figure out which databases, keywords, and other search criteria to use.
Accuracy	Look for evidence of research: Does the author refer to other work or research? Is there a list of references? Are the resources up to date? Are they from professional journals and similar publications? Look at how the facts are described: What kind of language, imagery and/or tone is used (e.g. emotional, objective, professional, etc.)? Is the information from a blog with opinions or from a peer-reviewed scholarly article? What kind of information is included in the resource? Is the content of the resource primarily opinion? Is it balanced (does it recognize and accurately present different views on the topic)? Does the author provide references or sources for data or quotations? Can you verify any of the information by looking in another source or from personal knowledge? Are there spelling, grammar or typographical errors?
Authority	Who is the author, what organization or publisher is responsible for the source? Are they experts on the topic they are writing about? (Find evidence of their organizational affiliations and other work.) Do they carefully attribute their information sources, do those sources support their work? What is the publisher's interest (if any) in this information? Are there advertisements on the website or adjacent to the text of the article? (Does the article seem to have been written to go with the advertisements?) Is there contact information, such as a publisher or email address? Does the URL reveal anything about the author or source? (.com .edu .gov .org .net, etc.) What sorts of other articles can be found at that URL?
Purpose	Why was this source written (e.g. to inform, teach, entertain, sell, persuade, anger)? Do the authors/sponsors make their intentions or purpose clear? Is the information written with emotion or facts? Are other sources or points of view missing from the author's work? How might the author's affiliation affect the point of view, slant, or potential bias of the source (does the author acknowledge their affiliations and political, ideological, cultural, religious, institutional, experiential, or personal biases, or try to obscure them)? Is the source fact or opinion or propaganda? Does the source, author, or publication, contain opinions about the proper interpretation of the facts? (If so, do they clearly separate their understanding of what the facts mean from what the facts are?) Is the creator/author trying to sell you something (a product, a belief, an opinion)?

Back to the top of the page