Questions About Cancer? 1-800-4-CANCER

Clear & Simple: Developing Effective Print Materials for Low-Literate Readers

  • Updated: 02/27/2003

Introduction

Many American Adults Lack Literacy Skills

According to the 1992 National Adult Literacy Survey, some 90 million American adults-- about 47 percent of the U.S. population-- demonstrate low levels of literacy. These individuals lack the literacy skills to function adequately in our increasingly complex society. Individuals with low or limited literacy may experience difficulty applying reading, writing, computational, and information processing skills to everyday life situations. While people with literacy problems may be found among all ethnicities, races, and classes, a significant correlation does exist between literacy, and both education and income levels.

Federal agencies often rely on print materials to tell people about health information and social services. Many of these materials are written at the 10th grade reading level and above. These materials are not useful to people with limited-literacy skills.

A workgroup composed of Government communicators from diverse Department of Health and Human Services' agencies met to address the disparity between the reading level of available print materials and the reading ability of many Americans. This set of guidelines, which was developed to assist writers in communicating effectively to low-literate audiences, is the result of their efforts.

About Clear and Simple

This guide outlines a process for developing publications for people with limited-literacy skills. The process was derived from communications, health education, and literacy research and practice. In addition, writers who have produced low-literacy materials contributed their expertise. Thus, the guide features both proven principles and a discussion of the real life issues that individuals developing low-literacy materials face, such as the constraints of time, budget, organizational pressures, and the Government publications process.

A review of existing examples and discussions with experienced writers makes it clear that the low-literacy field is an evolving discipline. Many definitions of "low-literacy" exist, and some products receive this designation incorrectly, simply because they test at a lower reading level. Even when the communicator pays careful attention to appropriate educational principles, the product may not exemplify perfectly these principles to all readers and reviewers. Deciding whether a passage or an illustration follows low-literacy guidelines and communicates effectively is a subjective judgment. This fact underscores the importance of testing all materials with the intended audience, whose understanding and acceptance is critical.

One writer summed up the challenge this way: "Everyone recognizes that we need to make print products more accessible to low-literate audiences. But developing low-literacy products is a major change from 'business as usual.' In some respects, it's like learning how to write, design, and test materials all over again."

Guidelines

The five standard steps in developing print materials are:

1. Define the target audience.

2. Conduct target audience research.

3. Develop a concept for the product.

4. Develop content and visuals.

5. Pretest and revise draft materials.


The following sections outline specific considerations for each
step in developing materials for readers with limited-literacy skills.

Step I: Define the Target Audience

A target audience is the group of people the communicator wants to reach with a message. People with limited-literacy skills compose a broad target audience, crossing all ethnic and class boundaries. However, there are some common characteristics among low-literate audiences regarding how they interpret and process information.
  • Tendency to think in concrete/immediate rather than abstract/futuristic terms.
  • Literal interpretation of information.
  • Insufficient language fluency to comprehend and apply information from written materials.
  • Difficulty with information processing, such as reading a menu, interpreting a bus schedule, following medical instructions, or reading a prescription label.
It is important to keep these characteristics in mind as you develop materials.

While the above characteristics provide some basic understanding of people with limited-literacy skills, it is essential to learn more about your audience as people, not just as statistics or generalizations. Target audiences may be defined by age, sex, marital status, educational level, occupation, income, religion, race, ethnicity, language, geographic location, lifestyle, health-related attitudes and behaviors, and many other characteristics. Understanding these factors is critical because audience characteristics influence each step in the process of developing low-literacy materials. The National Cancer Institute's (NCI) Making Health Communication Programs Work: A Planner's Guide (1989) provides further details on how you can define your target audience.




Questions You May Have About Defining the Audience

How do I decide whether I need to develop a low-literacy product versus a regular publication at a lower reading level?

The key criterion is a need to reach individuals with very poor or marginal written communication skills. These groups are distinct from general public audiences who may profit from materials written in simple language but who probably do not need special low-literacy-readers' aids and educational devices to comprehend a message.

A variety of factors can suggest a need to focus specifically on low-literate readers. One planner based her decision to target this group on an analysis of public knowledge of her topic, cancer prevention. Surveys repeated at 3-year intervals showed little-to-no gain in knowledge among some audiences despite ambitious public information campaigns designed at the 9th to 10th grade reading level. "That said to me that messages directed to the general public weren't getting through to all of the target audience," this program professional explained. "We decided to develop a prevention message specially targeted to low-literate Americans because demographic profiles and our experience had shown that this was a group our earlier campaigns could well have missed."

Another cancer prevention project also used available data to help identify its target audience. In this case, data showing that hard-core smokers were more likely to have lower educational levels argued for conveying a stop-smoking message in low-literacy format.

Once you decide that you need to reach readers with limited-literacy skills, then you can define that audience more specifically vis á vis your messages.

Can I communicate effectively to people with average or good reading skills by using a low-literacy format?

Testing your product with the audience is the only way to find out. The experiences and opinions of those in the field vary on this point. Some have found that better educated information seekers will not even pick up a product that clearly is aimed at a basic level. Others say that all readers appreciate a message conveyed simply and clearly and that those readers who want more detail can be directed to sources of more indepth information.

Step 2: Conduct Target Audience Research

Target audience research includes reviewing existing data and/or gathering new data to understand relevant physical, behavioral, demographic, and psychographic characteristics of your audience. This research can tell you: what the target audience already knows about your topic; what rumors, myths, and misinformation may exist about the topic; how audience members feel about the topic; and what questions and information gaps there are. Research also can help you define specific ethnic, cultural, and lifestyle preferences of your audience. This information is critical to developing culturally relevant materials, which are vital to reaching audiences at all literacy levels.

In conducting target audience research, the first task is to check existing sources of information, such as library databases; health statistics compiled by groups such as state and local health departments and the National Center for Health Statistics; Government or voluntary health organizations who have worked with your audience; and sources of polling information, such as polling companies. The box labeled "Information You Need" suggests the types of data to collect.


Information You Need About Your Audience

  • Age, sex, ethnicity, income and education levels, places of work, and residence.
  • Causative/preventive behaviors related to your topic.
  • Related knowledge, attitudes, and practices.
  • Patterns of use of related services.
  • Cultural habits, preferences, and sensitivities related to your topic.
  • Barriers to behavior change.
  • Effective motivators (e.g., benefits of change, fear of consequences, incentives, or social support).

In some cases, critical information about your audience will not be available in existing data. At this point, you may decide to conduct new research of your own to fill these information gaps. Whenever possible, supplement national data with local population statistics. National data may not capture unique characteristics of your audience.


Formal methods used to find out more about the target audience include:

Knowledge, Attitudes, and Practices Surveys

Surveys that measure the respondent's knowledge, attitudes, and practices on a specific topic are conducted by telephone, by mail, or face-to-face with members of the target audience.

Advantages: Provide highly targeted, directly relevant information; can provide estimates representative of the total population.

Disadvantages: Require time, statistical expertise, and resources to conduct; need a mechanism for locating and reaching large numbers of your target audience. Mail surveys are inappropriate for most low- literate readers.

Focus Groups

A facilitator, preferably one who has characteristics in common with the target audience (e.g., gender, race, ethnicity, shared experiences), conducts 1- to 2-hour group discussions with 6 to 10 representatives of the target audience.

Advantages: Require fewer audience representatives than other research methods; help explain why an audience feels or acts as they do; allow indepth discussion of issues; faster and can be less expensive than surveys.

Disadvantages: Findings cannot be projected to the population as a whole; findings are qualitative, rather than quantitative; require expertise in conducting, reporting, and applying results appropriately; need to locate and motivate members of your target audience to participate.

Audience Interviews

An interviewer conducts individual interviews with members of the target audience so that issues can be explored at length. These often occur in locations frequented by members of the target audience, such as clinics, community centers, Government service centers, literacy programs, and Adult Basic Education (ABE) and English as a Second Language (ESL) programs. They may be arranged by appointment or conducted with people who agree on the spot to participate.


Advantages: May yield more indepth information than focus groups.

Disadvantages: More time and labor intensive than focus groups.


For more information about quantitative and qualitative research methods, see Making Health Communication Programs Work: A Planner's Guide andDeveloping Health and Family Planning Print Materials for Low-Literate Audiences: A Guide.


Questions You May Have About Research

Must I use these research methods to develop an effective low-literacy product? I do not always have the time or the budget to do new audience research.

Many product developers echo this concern. While formal research methods can provide invaluable insights, many projects will not be able to make use of them due to a variety of practical constraints. The best approach is to make audience research a routine part of your product-development process for audiences of all literacy levels; then the time and budget for research will be built-in automatically.

What are the alternatives to formal research? Product developers commonly use two other methods to get needed information about their target audience: (1 ) seeking input from target audience members who agree to serve on an advisory board, or (2) seeking input from individuals who have close working contacts with the target audience.

"I've worked with this audience for 20 years," notes one writer, "and I have gained lots of insights. I feel confident in using this knowledge as a starting point. But I supplement it with advisory board guidance when I can't conduct formal research. And even when I can't do all the up-front research I would prefer, I know that pretesting will supplement what I've learned earlier and ensure that the product is on target."

When you rely on indirect information sources, such as asking health professionals what they believe their patients think, feel, or do, it is especially important to pretest the product with members of the audience themselves.

A low-literacy educator stresses that, "It's critical to get direct audience involvement at some point. No matter how well you or others think you know your target group, only someone with limited-literacy skills can provide a true test of your materials' comprehensibility and appropriateness. Programs that rely only on secondary or 'gatekeeper' opinions can make big mistakes."

If I do audience research before I develop my concept, how can I get information that is specific to my product?

The steps suggested here need not be completed in strict chronological order. In many cases, audience research occurs in several stages as product developers have need for new or different information.

Often before a concept statement is fully developed, you will have certain fundamental ideas in mind that could be tested during audience research. For example, you may have a key message line that you have always used in your products. Will it work with this audience? In producing a booklet on food safety, the FDA found that their traditional message, "keep hot foods hot, keep cold foods cold," did not communicate to low-literate readers. While the words are simple, they did not convey safety concepts clearly.

Another example might be the planning of a campaign or a product that includes a celebrity. In the initial research stage, you could determine whether the celebrities you have in mind are credible-- or known-- to this audience. One writer recalls the results of not taking this step. "I used to staff exhibits where we offered our products to health organizations with low-literate clients. Numerous program directors and nurses commented on one of our posters that used gambling odds-maker 'Jimmy the Greek' to convey a health message. They told us that their clients did not understand the concept at all. The target audience is response to the poster was 'Who's he?' or 'What does he have to do with health?' The program directors said the only way they could use the poster was to turn it over and write their own message on the blank side."

Step 3: Develop a Concept for the Product

Using the information gathered during audience research, you can begin to outline the objectives, style, format, and approach of the product that will carry your message. Many writers prepare a formal concept statement at this stage for all those involved in product development. Discussing the concept with individuals or groups who understand the needs of your target audience is an easy, inexpensive way to doublecheck the appropriateness of your intended approach. It is, however, no substitute for pretesting materials later with your target audience.

Leonard and Cecilia Doak, health and literacy experts, suggest five principles to follow when developing the concept for a low-literacy publication:
  • Define the behavioral objective(s) of the material.
  • Determine the key information points the reader needs to achieve the behavioral objective(s).
  • Select the most appropriate presentation method(s) (e.g., audio, audiovisual, print, radio, TV, interactive computer programs).
  • Decide on the reading level for the material if you select a print presentation.
  • Organize topics in the way the person will use them.


Define the Behavioral Objective(s) of the Material

To keep the product focused on concrete action, objectives should be behavioral rather than simply informational. For example, in designing a product intended to promote mammography among low-literate women, appropriate behavioral objectives might be:
  • The reader will ask the clinic or doctor about mammography at her next visit, or
  • The reader will make an appointment for a mammogram or
  • The reader will call us to discuss her options.
Objectives such as these would help guide content decisions to avoid unnecessary details. An objective that was purely informational (e.g., "The reader will understand why mammography is important to her health") would not necessarily lead to the development of the action-oriented messages low-literate audiences need.


Determine the Key Information Points the Reader Needs To Achieve the Behavioral Objective(s)

From the many possible content points, which ones will predispose and enable the reader to take the desired action?

The information you learned from audience research will help you make this determination. In addition, it is important to consider when and how a product will be used. The following questions are important to answer at this stage:

Will the material be used by the reader alone or interactively with a health or social services professional?


Will the product stand alone, or is it part of a series of materials, or of a broader communications initiative, such as a media campaign?
At what point in the learning process will the reader receive the product (e.g., regarding an illness: before diagnosis, after diagnosis, before or after a treatment decision, or before or after the reader is likely to have received related information)? What types of information will be useful and relevant (e.g., will this audience use a resource list of groups to write for more information)?

Select the Most Appropriate Presentation Method


At the most basic level, it is important to assess whether print is an appropriate medium for conveying your message to this audience. Will your audience pick up a print product, and, if so, under what circumstances? If they are used to relying on radio, TV, or word-of-mouth, can print materials serve their needs?

If you do choose print, will you create a l-page factsheet, a 10-page booklet, or three 1-page factsheets delivered at different points in the learning process? Budget and target audience information help shape such decisions. Two additional questions also are important to answer:

How will the product be distributed (e.g., in person, on a rack in a supermarket, through the mail)?


Is the material intended for one-time or long-term use?
Answers to these questions have important implications for the tone, structure, and design of the product.

Decide on a Reading Level for the Material

The term "reading level" refers to the number of years of education required for a reader to understand a written passage. Some experts suggest aiming for a level that is two to five grades lower than the highest average grade your audience achieved to account for a probable decline in reading skills over time. Others note that a third to fifth grade level frequently is very appropriate for low-literate readers.

How do you estimate reading level? Readability formulas often are used to assess the reading level of materials. Fry, Flesch, FOG, and SMOG are among the most commonly used readability formulas. Using these formulas is a simple process that can be done manually or by using a computer software program. Each method takes only a few minutes.

Typically, readability formulas measure the difficulty of the vocabulary used and the average sentence length. In addition, the computer software programs analyze a document's grammar, style, word usage, and punctuation, and assign a reading level. However, these formulas do not measure the reader's level of comprehension. Grammatik, and other readability software programs are available from computer stores or from the manufacturer. Note: Mention of software products does not constitute an endorsement by the National Cancer Institute.

Reading levels and readability formulas are useful aids in targeting publications to an audience. Yet none of the readability formulas listed above were designed to be used as writing guides. According to researchers James Pichert and Peggy Elam, most were designed originally to rank the difficulty of books to be used at a specific grade in school. Thus, using such formulas in product development is no guarantee of producing well-written, understandable text. In fact, their creators acknowledge that using the formulas as writing guides can have serious negative consequences. For example, one writer is concerned that the formulas' emphasis on short sentences and short words may produce a choppy text that leaves out familiar terms because they are polysyllabic. In addition, some formulas were tested on children rather than on adults, limiting their applicability to adult low-literate readers.


Pichert and Elam suggest three principles for using readability formulas effectively:

  • Use readability formulas only in concert with other means of assessing the effectiveness of the material.
  • Use a formula only when the readers for whom a text is intended are similar to those on whom the formula was validated.
  • Do not write a text with readability formulas in mind.

For more information on readability formulas, see Teaching Patients With Low- Literacy Skills and Making Health Communication Programs Work: A Planner's Guide.

Organize Topics in the Way the Person Will Use Them

Give readers the most important points first and last; studies show that low-literate audiences remember these best. Literacy experts also suggest grouping information into "chunks," with a clear, ordered format. You may sequence these as steps (1,2,3), chronologically (by time of day), or by topical arrangement (main heading, subheadings) depending on how the person will use the information.

It also is useful to give the reader an action step he or she can do right away (e.g., call your clinic, send in a request). This reinforcing action tends to improve retention of information and moves the reader into desired behaviors immediately.

Questions You May Have on Concept Development

In trying to limit my publication to a few key points, how do I determine what information the reader absolutely needs and what I can leave out?


Focusing on your behavioral goals for a publication can help you decide if an information point is fundamental and helps to motivate a desired action.

One writer separates information points into those the reader needs to know, those the reader might (or might not) want to know, and those the sponsoring agency wants to tell. "Most of the time, information that falls in the second two categories can be eliminated. I try to make my inclusion criterion, 'Does the reader need this statement, fact, or reader's aid to understand, accept, and take the desired actions?' This cuts out a lot of detail, but some tough decisions always remain. I try to make sure that pretesting particularly addresses these points, because it's the only way to determine the information that the audience really needs."

Writing at the correct reading level for the audience is important, but I also am hearing that I should not write my text with a readability formula in mind. What should I use to help me write at an appropriate level?

Most writers intrinsically shy away from formulas. Yet, because good writers are highly literate themselves, they may not trust their usual instincts to tell them what will work for people with limited reading skills.

The checklist in the next section outlines the fundamental principles of writing and designing a low-literacy publication. These guidelines address the readers' needs and should provide a basic framework within which a writer can operate comfortably.

One low-literacy-materials writer illustrates the value and limitations of readability formulas. "I made my written explanation as simple and action-oriented as I could. Then I put it down for a while and came back to it with a critical eye. I rewrote it several times until I was happy with the flow and the comprehensibility of the text. At this point, I used several readability formulas to see what level I had achieved.

When the product tested at the sixth grade level, I was surprised. I went back and found I could simplify some of the text without detracting from my message. It ultimately tested at the 4th grade level-- and pretesting still showed us that one or two concepts were not well understood."

While readability formulas and software do not produce good low-literate materials, they can be useful in approximating a reading level or, in the case of some software products, identifying specific problems that inhibit readability.

Step 4: Develop Content and Visuals

Once you have agreed on a product concept, you can begin outlining and writing the first draft. The content information you include will be based in part on the audience needs and interests identified in your research.

In preparing your draft, it will be important to tailor content, layout, and the use of visuals to the needs of a reader with poor reading and communication skills. What does this mean in concrete terms? Low-literacy experts have identified key principles for developing effective materials for this audience. These principles are summarized in the checklist below. You can use this list as you are developing a new publication and as a doublecheck in reviewing product drafts.

Checklist

  • Key Principles of Effective Low-Literacy Print Materials
  • Examples of Effective Communication
Many organizations have developed products that illustrate effective use of the checklist guidelines. This section discusses each communication principle on the checklist in more detail and provides examples of their use in existing products. A complete listing of products can be viewed by clicking on the following links:

Content/Style

  • The material is interactive and allows for audience involvement.
  • The material presents "how-to" information.
  • Peer language is used whenever appropriate to increase personal identification and improve readability.
  • Words are familiar to the reader. Any new words are defined clearly.
  • Sentences are simple, specific, direct, and written in the active voice.
  • Each idea is clear and logically sequenced (according to audience logic).
  • The number of concepts is limited per piece.
  • The material uses concrete examples rather than abstract concepts.
  • The text highlights and summarizes important points.

Layout

  • The material uses advance organizers or headers.
  • Headers are simple and close to text.
  • Layout balances white space with words and illustrations.
  • Text uses upper and lower case letters.
  • Underlining or bolding rather than all caps give emphasis.
  • Type style and size of print are easy-to-read; type is at least 12 point.

Visuals

  • Visuals are relevant to text, meaningful to the audience, and appropriately located.
  • Illustrations and photographs are simple and free from clutter and distraction.
  • Visuals use adult rather than childlike images.
  • Illustrations show familiar images that reflect cultural context.
  • Visuals have captions. Each visual illustrates and is directly related to one message.
  • Different styles, such as photographs without background detail, shaded line drawings, or simple line drawings, are pretested with the audience to determine which is understood best.
  • Cues, such as circles or arrows, point out key information.
  • Colors used are appealing to the audience (as determined by pretesting)

Readability

  • Readability analysis is done to determine reading level.


Questions You May Have About Content Development

Technical terms raise the reading level of a publication. Do low-literate readers need to know them? Will they understand them?

There is no absolute answer to this question. Pichert and Elam point out that when technical terms are used in the reader's daily life, they can be as familiar as any other words. For example, an older American will know the terms "Social Security benefits," and a person with diabetes will be accustomed to technical words such as "insulin" and "reaction."

When a text does include technical terms, some writers suggest putting a simple explanation next to the term. They believe a glossary approach may add to the low-literate reader's difficulty in getting through the text. People may not realize that unfamiliar words are defined in a separate section.

Many practitioners feel that the "need to know" criterion is an important one to apply to this decision. "If my objective is to increase the use of mammography, I have to use that term," one writer says. "But if I'm asking people with HIV to take AIDS - delaying actions, does the audience need to know the term 'opportunistic infection'? I chose to avoid the term, and pretesting showed that the audience understood what they needed to do to avoid infections without learning it."

How can I keep material on technical topics simple throughout an expert and/or organizational review process?

Every writer interviewed for these guidelines had experienced this concern. The simplicity of effective low-literacy products is startling to many reviewers, especially those accustomed to sci- entific or technical publications. They often are unfamiliar with low-literacy techniques and may be concerned that a product written at a low reading level may reflect poorly on the expertise of an agency or organization.

Several writers suggested ways to work constructively with reviewers on this issue. Ideas include:
  • Educate reviewers about the need and scientific foundation for low-literate writing techniques.
  • Involve reviewers at the concept development stage so that they are not surprised at the draft they receive.
  • Make sure that all simple explanations are accurate. Do not distort the scientific or technical facts as you pare away the details.
  • Work personally with the reviewers. If a suggested change is inappropriate, discuss both of your concerns and work cooperatively toward a solution.
  • Test reviewer-inserted concepts, specifically during prepublication evaluations. If the reviewer's idea does not work with the audience, you will have a firm basis for change.

Are pictorial signs, symbols, and charts more effective than words for a low-literate reader?

Not necessarily. Some experts suggest that "universal" symbols, such as a stop sign, an arrow, or a big black "X" usually test well with this audience. When a pictorial representation is open to interpretation, however, it can fail to communicate with any audience. Likewise, while a simple chart may work well, a large matrix or visually busy schema are likely to confuse. For example, functionally illiterate individuals have trouble using a bus schedule.

I know that low-literate products should focus only on a few key concepts. How do I handle a complex topic with 8 or 10 important messages when I can only afford to do one low-literacy publication?

A strong grouping of main and subpoints is a common solution to this problem. When individual sections are sequenced effectively and each can stand alone, readers can approach the text at their own pace.

I cannot afford to do separate low-literacy publications for all of our organization's publications. Is there an effective way to adapt higher level reading materials for low-literacy populations?

For many years, the only products available to professionals who work with low-literate groups were written at a high reading level. These professionals became adept at picking out the key concepts and highlighting them for their clients-- using underlining, circles, stars, or arrows that meant "pay special attention to this."

Although the experts would not call this approach ideal, it does meet the readers' needs better than untailored higher level material.

Writers also attempt to meet the needs of both audiences by using headlines and subheads to carry key message points, in logical order. Low-literate readers-- and others who only skim written materials-- can skip the details that the accompanying text provides.

All of my products need to be photocopied and that means I cannot use color. Sometimes my budget will not allow for illustrations either. Can I still design an effective low-literacy product?


Although color is a powerful communication tool, strong format, good use of white space, and alternative highlighting devices can help a black-and-white product convey its message.

Low-literacy products do not always need illustrations to "break up the text." Boxes, lines, and white space can keep a design from being too copy-dense.

Pictures that illustrate an action or a key point are valuable, however, and they do not have to be expensive to produce. In fact, simple line drawings usually are preferable to detailed pictures for this audience. Even "off-the-shelf" computer clip art can be effective if it fits the message and tests well with the audience.

Step 5: Pretest and Revise Draft Materials

Pretesting is a qualitative measure of audience response to a product. As previous sections have suggested, it is critical to pretest draft messages and visuals with members of the intended audience. Pretesting helps ensure that materials are well understood, responsive to audience needs and concerns, and culturally sensitive. Leonard and Cecilia Doak discuss pretesting, which they call "learner verification," as a tool for identifying elements in a communication that need to be changed to make it more appropriate for the intended audience.

Although funds may not be available for extensive pretesting, some pretesting is essential to ensure that materials are culturally relevant and understandable to the target audience.

Pretesting is an important step in developing materials for any audience, and references such as Making Health Communication Programs Work explain general principles and provide "how-to" information. The focus of this section, however, is the special requirements for pretesting low-literacy materials.




Pretesting Variables: What To Test for


According to Leonard and Cecilia Doak, the basic purpose of pretesting materials with low-literate audiences is to find out whether the readers understand your messages. In addition to comprehension, three other factors are important: audience attraction to the product, its acceptability to them, and their personal involvement with the material. In discussing materials with participants, key issues to probe for include the following:

Comprehension

Does the respondent understand what the material is recommending and how and when to do it? Is anything unclear, confusing, or hard to believe? What meaning does the respondent attach to key words? to symbols and abbreviations? to visuals? Important aspects include:
  • Suitability of the words used: "What does the educational piece mean when it says to eat balanced meals? How do you do that?"
  • Distinguishing key details: "Which vegetables have lots of fiber?"
  • Meaning or relationship of visuals to text: "Looking at this picture, how will you cut down on fat in your soups or stocks when cooking?"

Attraction

What kind of feelings does the material generate-- enthusiasm? just OK? or a "turnoff"? For example, "Are the people in the material attractive to you? Is there anything you don't like about the people (or pictures) in this material? How about the color and the layout of the material?"

Acceptability

Is the material compatible with local culture? Realistic? Would it offend people in any way? Are the hairstyles, clothing, etc., appropriate?
  • Suitability for both sexes and all ages: "Is this mostly for men (women)?"
  • Supportive of ethnic practices: "Do you think your friends and neighbors would be willing to cook their foods this way?"
  • Personal Involvement or Relevance. Can the respondent see himself or herself carrying out the actions called for in the materials?
The Doaks also stress that not every word, picture, or idea has to be tested. The most important points to verify are that the reader understands what he/she should do and how to do it. Comprehension often depends on understanding fundamental vocabulary terms and visuals. For example, the word "diet" may be understood by different people to mean different things; applying the wrong definition could lead to the wrong behavior. When key content is conveyed in word/picture association or tables/lists, it is critical to test comprehension of these formats and to ensure that the reader has the skills needed to use them.



Organizing a Pretest


Pretesting with low-literate audiences involves some unique logistical considerations. This section discusses some issues to keep in mind while organizing your pretest.

When to Pretest

At either or both of these stages of product development:
  • Rough draft of copy and graphic concepts using a manuscript of the product, with a few examples of potential illustrations.
  • Preliminary typeset, laid-out version of the product with rough graphics in place.

Where to Pretest

  • Clinic/hospital waiting rooms.
  • Doctor's office.
  • Client's home.
  • Community facilities (e.g., church, senior center).
  • Adult Basic Education and English as a Second Language classes.
  • Agency/organization's facilities (e.g., Social Security office, WIC center, job training centers).

Environment

Some researchers believe it is best to test a product in the same type of environment in which a reader will be using the material. For example, if a patient will be reading a factsheet in a noisy, busy clinic, be sure that test readers have the same distractions.

Methods

  • Individual interviews (10-20 minutes each)
Advantages: Provide "cleanest" results; less chance of one respondent biasing another; good for short materials.

Disadvantages:
Scheduling may not allow speedy completion.
  • Group Interviews (8 to 10 people, 30-60 minutes)
Advantages: Better for longer pieces (booklets, kits); group discussion may elicit valuable information not contained in interview questions.

Disadvantages:
Need to organize in advance; need a trained group facilitator to conduct. Session must be paced to maintain attention.

Number of Participants

Ideally, at least 25 to 50 members of the intended audience should review your product. If time or budget is limited, however, it is far better to test the material with 5 to 10 people than not to test it at all.
You can feel comfortable with fewer participants if the product is:
  • Very simple.
  • Very short, needing only a brief attention span to complete.
  • Directed at one homogeneous target group.
You need more participants if the product is relatively:
  • Complex.
  • Long, demanding a sustained attention span.
  • Culturally diverse.


Determining the Reading Level of Pretest Participants


Are you pretesting your materials with the right readers? Having pretest participants who have the same characteristics as the low-literate audience you are trying to reach is critical to the validity of your pretest results. Recruiting participants through groups or settings that include people with limited-literacy skills is a logical starting point. But the only way to be sure your pretest volunteers read at the same level as your intended audience is to test their reading skills. The Wide Range Achievement Test (WRAT) is used to measure reading levels and the Cloze technique is used to measure comprehension. To avoid offending or causing discomfort to those whose reading ability you are testing, you can integrate a WRAT or a Cloze test into the pretest interview. For example, in a recent pretest conducted by the National Cancer Institute, the interviewers introduced the WRAT test as the last part of the pretest. They stated, "Thank you for helping with the questions on the chemotherapy booklet. We need your help with one last part-- a word list. This will take only a few minutes. The word list will help us know how difficult the words are in the chemotherapy booklet." This integrated approach spared the participant the pressure or potential embarrassment of "failing a reading test."

The Wide Range Achievement Test

The Wide Range Achievement Test is based on word recognition and does not measure comprehension or vocabulary. An efficient way to determine reading levels, it takes only a short time to administer.

The WRAT has the reader look at a written word and say that word out loud. Pronouncing the word correctly shows that the reader recognizes the word. The WRAT focuses on recognition because, at the most basic level, if a person does not recognize a word, comprehension is impossible.

The test involves listening to the participant read from a prepared list of words, arranged in increasing order of difficulty (see sample list below). The test is over after the reader mispronounces ten words, and the test administrator notes the level at which the last mispronunciation occurred. The "stop" level equates to a grade level of reading skills. You can compare this level with the reading level of your intended audience to see if your pretest readers are a good representative "match." The Low Literacy Publications Section at the end of this guide provides contact information for obtaining WRAT testing instruments.


The Cloze Technique

This procedure measures the reader's ability to comprehend a written passage. Because it requires readers to process information, it may take up to 30 minutes to administer.

In a Cloze test, text appears with every fifth word omitted. The reader tries to fill in the blanks. This task demonstrates how well he/she understands the text. The reader's ability to identify the correct word also reflects his/her familiarity with sentence structure.

While packaged Cloze tests are available, Leonard and Cecilia Doak's Teaching Patients With Low Literacy Skills explains how to make up and score a Cloze test yourself, based on the materials you are pretesting. Their book also discusses use of the WRAT to assess reading levels.


Conducting Pretest Interviews: Special Considerations


Just as effective low-literacy products are designed to meet audience needs, effective pretesting also must be tailored to participants with limited-reading skills. Those experienced in pretesting with a low-literate audience offer the following suggestions:
  • Give only one product to each individual or group even if you are testing a series of publications. This ensures the respondent's maximum concentration level for each product.
  • Distance yourself from the product and assure participants that you want their honest assessment. "If you don't," one pretester explains, "this audience is more likely than higher level readers to avoid criticizing. Perhaps it's because they aren't sure they understand or perhaps they aren't comfortable explaining negative responses."
  • Make sure participants understand that they are not being tested, the materials are. Explain this clearly when you introduce the materials and the participant's role. People concerned about their literacy skills need this reassurance.
  • Pay attention to audience interest in the proceedings. Change the pace, if necessary, to maintain attention levels. When attention was waning in a large group discussing product text, for example, the facilitator split the group in half to allow more one-to-one interaction and switched the focus to illustrations.
  • Choose people to recruit and interview pretest participants who are culturally sensitive and who have good social skills. Unless potential participants feel at ease with the interviewing staff, they may not agree to take part.



Using Pretest Results To Revise Materials


Pretest results, usually in the form of a report with recommendations, are not a blueprint for revisions. They raise issues and point out problems; the solutions are up to you. Common issues about low-literacy products include the following:

Should the product be scrapped and begun again when pretesting raises concerns?

Much of the time, simple to extensive revisions can "fix" problems that pretesting uncovers. You should start over only when the majority of responses indicate fundamental problems, such as:
  • Most readers were completely lost and could not identify the key behavioral actions the material was designed to convey.
  • The medium interferes with the message (e.g., a large "chart" poster is too complicated to understand).
  • The format is altogether unappealing or off target (e.g., readers do not want a booklet, they want a video; a tip sheet gives only "how-to" information, but readers really want background information).
  • The product is culturally inappropriate, with no relevance to the audience.

If one or only a few respondents raise a concern, are revisions necessary?


This is a judgment call. Your answer will depend on:
  • How many respondents you had. If you only tested the material with 10 people, each response needs to be considered seriously; with 50 participants, percentages can be relied on more comfortably.
  • The nature of the comment. Any remark that shows lack of understanding of a key concept should receive careful attention. If one person did not grasp a point, others may have problems as well. Idiosyncratic comments about a product's appeal or personal relevance are less of a concern, however. No format will please everyone in any audience.

Should respondents' suggestions for revising a product be followed?

Pretest participants are "experts" in what they can understand and accept in a product; they are not experts in materials' design. Most of the time, professional judgment is needed to devise an effective way to address reader concerns.

Many writers like to attend pretest group sessions or read interview questionnaires themselves rather than relying only on the pretest report. To get the most useful results, writers often contribute to questionnaire development as well. That way, they can draw special attention to specific wording or format choices they want an audience to validate.




Questions You May Have About Pretesting

My timeframe is very tight. How can I fit pretesting into the schedule?

Pretesting need not be an elaborate, time-consuming research project. How long it takes depends on how quickly you work, how elaborate your internal review process is, how large your test group sample is, and the schedules of the sites or organizations involved. The best approach is to include time for pretesting in the initial product development schedule. If time is short, plan a small-scale test and begin the preparatory steps right away. Much of the preliminary work in pretesting can be completed long before the actual testing period. For example, arranging with a group or site to get access to audience representatives can begin during concept development. You can begin to draft your questionnaire while a product is still going through internal review, with any needed modifications made later.
"At first I had to educate my organization about the critical role pretesting plays in low-literacy product development," one writer remembers. "Now that everyone has seen how valuable it is, pretesting is no longer a 'refinement' we consider using in special circumstances. We make time for it in every low-literacy product development process."

How can I get access to people from the target audience for pretesting?

Many agencies work with the literacy organizations listed in the Resources section of this guide. State directors of Adult Basic Education and English as a Second Language programs often are good sources.


You also can cooperate with groups that represent your particular target audience. In pretesting booklets about HIV and AIDS, for example, the National Institute for Allergy and Infectious Diseases (NIAID) worked with organizations such as People With AIDS and community clinics that are part of NIAID's research network. Several professionals interviewed for this guide suggested that offering incentives, such as small cash payments, or reduced waiting time for an appointment, can help motivate individual participation. Overworked and underfunded literacy programs also are more able to provide timely assistance if your agency can compensate them for their recruiting time.

Professional intermediaries distribute our pamphlets to their clients. Should they be part of pretesting?

Professional intermediaries and advisory groups can be an important source of insights about your product's utility. Product developers interviewed for this guide cautioned, however, that responses of audience members should take precedence over professional views if you receive conflicting opinions. It also is important not to rely on professional review alone; no matter how close professionals are to their clients, they are not low-literate themselves.

References

1. Kirsch I, Jungeblut A, Jenkins L, Kolstad A. Adult literacy in America: A first look at the results of the National Adult Literacy Survey, National Center for Education Statistics, Department of Education, 1993

2. Dowling W. Reading level analysis of written materials distributed by the Ohio Cancer Information Service, Columbus, Ohio, Adult Literacy Center for Individual and Organizational Development, 1990

3. Zimmerman M, Newton N, Frumin L, Wittet S. Developing health and family planning print materials for low-literate audiences: A guide. Washington, D.C.: Program for Appropriate Technology in Health, 1989

4. US Department of Health and Human Services. Making health communication programs work: A planner's guide. Bethesda, MD: National Cancer Institute, Office of Cancer Communications, 1989

5. Doak C, Doak L, Root J. Teaching patients with low literacy skills. Philadelphia: JB Lippincott, 1985

6. Pichert JW, Elam P. Readability formulas may mislead you. Patient Educ Couns 1985;7(13):
181 -91

7. Gatson N, Daniels P. Guidelines: Writing for adults with limited reading skills. Washington, D.C.: United States Department of Agriculture, Food and Nutrition Service, 1988

8. Jonassen D, ed. The technology of text: Principles for structuring, designing and displaying text. Vol 2. Englewood Cliffs, NJ: Educational Technology Publications, 1985

9. Bertrand, Jane E., Communications Pretesting, Chicago: Community and Family Study Center, University of Chicago, Media Monograph 6, 1978

10. Jastak Association, Inc. Wide Range Achievement Test (WRAT). Copyright 1984

11. Taylor, WS. Cloze procedure: A new test for measuring readability. Journalism Quarterly 30:415-433. Fall, 1953

Appendix 1: Content/Style


  • The material is interactive and allows for audience involvement.
Publications designed to produce behavioral change need to help readers get involved with the information and take concrete action. A variety of devices can help make a product an interactive educational tool.

Example from How to Take Care of Your Baby Before Birth gives the reader a contact for more information.

Example: How to Take Care of Your Baby Before Birth.



  • The material presents "how-to" information.
Buying Food and Eating Out gives concrete advice for making the behavior change the booklet suggests: cutting down on salt.

A Mammogram Could Save Your Life tells the reader how to find out about where to get a mammogram.

Example One: Buying Food and Eating Out

EATING AWAY FROM HOME

Today people eat lots of foods away from home-- at school, in restaurants, and other places. When you eat out, remember the food will usually already have salt in it, so don't add more. Using sauces, gravies, salad dressings, pickles and seasonings such as catsup, mustard, and soy sauce adds more salt to foods. Leaving these off can make quite a difference! Many restaurants, if you ask them, will cook plain foods without adding salt.
Teenagers often go to fast food restaurants. Some of the foods may have lots of salt. But you can cut back when you eat there, too. You can:
  • Order your sandwich or burger plain without the catsup, mustard and pickles, or try to leave at least one off.
  • Eat your French fries and other foods without putting salt on them.
  • Learn which foods contain more sodium. For example, burgers with cheese contain more sodium than plain burgers. Milkshakes contain more sodium than plain milk, French fries without salt are low in sodium, and orange juice has almost no sodium. Several fast food restaurant chains know how much sodium is in their different foods and can give you this information. Ask for it.
______________________________________


Example Two: A Mammogram Could Save Your Life




  • Peer language is used whenever appropriate to increase personal identification and improve readability.
  • Words are familiar to the reader. Any new words are defined clearly.
  • Sentences are simple, specific, direct, and written in the active voice.
Each of the examples features short sentences in the active voice. The pieces rely on peer language, using common words such as "having sex," "germs," and "causes death instead of technical terms such as intercourse, bacteria, and mortality. When a booklet does introduce a technical term-- "menopause," it appears in parenthesis after the more commonly used "change of life."

Example One: If You've Ever Had VD, Learn About AIDS



___________________________________________________________________


Example Two: Keep Your Food Safe

Wood cutting boards are very hard to clean.
Germs hide in the cracks. Use a plastic cutting board instead.

___________________________________________________________________


Example Three: The Pap Test: It Can Save Your Life!

Who needs to have a Pap test?
You do if:
- You are over 18; or
- You are 18 or under and have sex
There is no upper age limit for the Pap test.
Even women who have gone through the change
of life (menopause) need a Pap test every year.



  • Each idea is clear and logically sequenced (according to audience logic).
  • The number of concepts per piece is limited.

Having a Pelvic Exam and Pap Test
uses arrows to sequence the steps of the pelvic exam, thus preparing the reader for what to expect from the exam. The booklet limits the number of concepts by focusing on the pelvic exam.


Example: Having a Pelvic Exam and Pap Test


  • The material uses concrete examples rather than abstract concepts.
These passages tell readers what the terms "run in families" and "cooking until done" mean in concrete terms.

Example One: Diabetes and American Indians

Does high blood sugar run in families?


Yes. Brothers, sisters and children of people with
diabetes tend to get high blood sugar.

You can control blood sugar.

_________________________________________________________________


Example Two: Keep Your Food Safe

Raw meat, raw poultry, raw seafood, and raw
eggs can make you sick. Cook them until they are done:



  • The text highlights and summarizes important points.


The Pap Test: It Can Save Your Life uses color, large type, and underlining to highlight and summarize the main message.

Example: The Pap Test: It Can Save Your Life


Appendix 2: Layout


  • The material uses advance organizers or headers.
  • Headers are simple and close to the text.

Keep Your Food Safe divides recommendations into two categories that organize safety advice. Headers highlight the categories.

Subheaders on individual pages also are effective reader aids. The subheads in Having a Pelvic Exam and Pap Test help group the concepts.

Example One: Keep Your Food Safe


________________________

Example Two: Having a Pelvic Exam and Pap Test


  • Layout balances white space with words and illustrations.
  • Headings use upper and lower case letters.
  • Underlining or bolding, rather than all caps, give emphasis.
  • Type style and size of print are easy-to-read; type is at least 12 point.

This example from The Pap Test: It Can Save Your Life!, shows how good use of white space can make a page easy-to-read and approachable for poor readers.

Experts have learned that using all capital letters as an attention-getting device is not effective with low-literacy audiences because the all caps style is harder to read. Large, easy-to-read type also is important for legibility.

The Pap Test: It Can Save Your Life! makes the task of reading as easy as possible. It highlights by using color, bolding, and underlining, and has a very legible 14 point typeface. Use of white space also is liberal.


Example: The Pap Test: It Can Save Your Life!

Appendix 3: Visuals


  • Visuals are relevant to the text, meaningful to the audience, and appropriately located.
  • Illustrations and photographs are simple and free from clutter and distraction.
  • Visuals use adult rather than childlike images.
  • Visuals have captions. Each visual illustrates one message.
While this series of illustrations is easy to understand, it does not insult the reader by appearing cute. The drawings also avoid seeming childlike by crediting the reader's adult life experience in the kitchen.

Example: Keep Your Food Safe


  • Illustrations show familiar images that reflect cultural context.
  • Different styles, such as photographs without background detail, shaded line drawings, or simple line drawings, are pretested with the audience to determine which is understood best.

This idea is especially important for low-literacy publications targeted to ethnic cultures.

In the first example, from Diabetes and Native Americans, the illustration shows a traditional Native American theme. This picture accompanies text discussing the negative effect on Native American health of a change to modern lifestyles. A Native American artist prepared the visuals.

The second example, shown on the next page, targets the booklet to an Hispanic audience. A simple line drawing is used to illustrate the booklet.

Example One: Diabetes and Native Americans

Changes

Diabetes is a major health problem for Native American adults. It is mainly due to changes in eating and exercise.

Elders can recall times when people gathered and hunted food for simple meals. People walked a lot.

Now, we buy food at stores. We drive cars rather than walk to places.


_______________________

Example Two: Cómo cuidar a su hijo antes del nacimiento


  • Cues, such as circles or arrows, point out key information.
Visuals that draw the reader's eye to important points are an effective form of emphasis. The Pap Test, for example, uses an arrow to make sure readers move on to the next page; the device has been shown effective for this purpose in a number of pretesting experiences. If You've Ever Had VD, Learn About AIDS, puts circles by its core messages.

Example One: The Pap Test: It Can save Your Life!


_______________________________________

Example Two: If You've Ever Had VD, Learn About AIDS


  • Colors used are appealing to the audience (as determined by pretesting).
Although much of an audience's response to color is idiosyncratic, some have an objective basis. Bright colors, for example, often get the most attention because they are the most noticeable. Some colors or color combinations are harder to see; older people, for instance, often have difficulty seeing words or shapes that are yellow. Dark type usually is easier to read than light-colored type, and many have trouble reading type that is "reversed," light-colored letters on a dark background color.

Example One: Keep Your Food Safe



_________________________

Example Two: The Pap Test: It Can save Your Life!

Appendix 4: Low-Literacy Publications and Software

Publications

Adult functional competency: A summary. University of Texas at Austin, 1975. To order, contact: University of Texas at Austin, Office of Continuing Education, Austin, TX 78712

Allensworth DD, Luther CR. Evaluating printed materials. Nurs Educ 1986; 11 (2):18-22

American Cancer Society. Cancer and the poor: A report to the Nation. 1989. To order, contact: American Cancer Society, 1599 Clifton Road, N.E., Atlanta, GA 30329 (404) 329-7634

Beyond the Brochure: Alternative Approaches to Effective Health Communications may be available from the Centers for Disease Control and Prevention (CDC). To order, contact: CDC, Technical Information Services Branch, MSA34, Atlanta, GA 30333; (770) 488-5080

Brockett RG. Developing written learning materials: A proactive approach. Lifelong Learning 1984;7(5):16-18, 28

Brown H. Literacy training and older Americans. American Association of Retired Persons, 1989. To order, contact: AARP, Public Policy Institute, 601 E Street, N.W., Washington D.C. 20049; (202) 434-2277

Centers for Disease Control. Literacy and health in the United States: Selected annotations. Atlanta, GA.1991. To order, contact: Centers for Disease Control, Center for Chronic Disease Prevention and Health Promotion, Technical Information Services Branch, MS A34, Atlanta, GA 30333; (770) 488-5080

Childers T. The information-poor in America. Metuchen, NJ: Scarecrow Press, 1975

Crawford AN. A Spanish language Fry type readability procedure: Elementary level. Los Angeles: Bilingual Education Paper Series, Evaluation Dissemination and Assessment Center, California State University, Los Angeles, 1984;7:1-17

Doak C, Doak L, Root J. Teaching patients with low literacy skills. Philadelphia: JB Lippincott,1985

Fox M. A look at illiteracy in America today-- the problem, the solution, the alternatives. Washington, D.C.: Push Literacy Action Now, 1986. To order, contact: Push Literacy Action Now, 1528 Pennsylvania Avenue, S.E., Washington, DC 20003; (202) 547-8903

Garcia WF. Assessing readability for Spanish as a second language: The Fry graph and Cloze procedure. Unpublished doctoral dissertation, Teachers College, Columbia University,1976

Gatson N, Daniels P. Guidelines: Writing for adults with limited reading skills. Washington, D.C.: United States Department of Agriculture, Food and Nutrition Service, 1988

Gilliam B, Pena SC, Mountain L. The Fry graph applied to Spanish readability. The Reading Teacher 1980;33:426-30

Hunter CJ, Harman D. Adult illiteracy in the United States. A report to the Ford Foundation. New York: McGraw-Hill, 1979

Wide Range, Inc. Wide Range Achievement Test (WRAT). Copyright 1984. To order, contact: Jastak Association, Inc., P.O. Box 3410, Wilmington, DE 19804-0250; (302) 652-4990 or (800) 221-9728

Jonassen D, ed. The technology of text, principles for structuring, designing and displaying text. Vol 2. Englewood Cliffs, NJ: Educational Technology Publications, 1985

Kirsch I., Jungeblut A. Literacy: Profiles of America's young adults, final report. Princeton, NJ: National Assessment for Educational Progress, 1986

Kirsch I., Jungeblut A, Jenkins L, Kolstad A: Adult literacy in America: A first look at the results of the National Adult Literacy Survey, National Center for Education Statistics, Department of Education, 1993

Kominski R. Educational attainment in the United States: March 1989 and 1988. US Department of Commerce, Economics, and Statistics Administration, Bureau of the Census, 1991

Kozol J. Illiterate America. Garden City, NY: Anchor Press,1985

Matiella AC. Getting the word out: A practical guide to AIDS materials development. Santa Cruz: Network Publications, 1990

Melton RJ. Developing and adapting written materials for the low literate hypertensive. Salinas, CA: Monterey City Department of Health,1990

Mettger W. Communicating nutrition information to low-literate individuals: An assessment of methods. National Cancer Institute, 1989. To order, contact: Office of Cancer Communications, NCI, Building 31, Room 10A03, 9000 Rockville Pike, Bethesda, MD 20892; (301) 496-6667

Michielutte R, Bahnson J, Beal P. Readability of the public education literature on cancer prevention and detection. J Cancer Education 1990;5:55-61

Nelson GD, Nelson B. Are your patient education materials readable? Health Educator Nov/Dec 1985;3(6):10-11.

Nitzke S, Shaw A, Pingree S, Voichick J.- Writing for reading: A guide for developing print materials in nutrition for low-literacy adults. Department of Agricultural Journalism, University of Wisconsin-Madison, 1986. To order, contact: Human Nutrition Information Service, Nutrition Education Division/GERB, Federal Building, Room 353, 6505 Belcrest Road, Hyattsville, MD 20782; (301) 436-5194

Ontario Public Health Association and Frontier College. The literacy and health project-- phase one. 1989. To order, contact: The Ontario Public Health Association, 468 Queen Street, E, Suite 202, Toronto, Ontario M5A lT7; (416) 367-3313

Pichert JW, Elam P. Readability formulas may mislead you. Patient Educ Couns 1985;7(13):181 -91

Rasmuson M, Seidel R, Smith W, Booth E. Communication for child survival. Academy for Educational Development, 1988. To order, contact: AED, 1255 23rd Street, N.W., Washington, D.C. 20037, Attention: Heathcom; (202) 884-8700

Research and Training Center on Independent Living. Guidelines for reporting and writing about people with disabilities. 1990. To order, contact: Media Project, Research and Training Center on Independent Living, Bureau of Child Research, University of Kansas, 4089 Dole Building, Lawrence, KS 66045; (913) 864-4095 (voice/TDD)

Rice M, Valdivia L. A simple guide for design, use, and evaluation of educational materials. Health Educ Q 1991; 18 (1):79-85

Ross Laboratories. Nutrition education opportunities: Strategies to help patients with limited reading skills. Second Ross Roundtable on Current Issues in Public Health, 1988

Smith SB, Alford BJ. Literate and semi-literate audiences: Tips for effective teaching. J Nutr Educ 1988;20(5)

Spaulding S. A Spanish readability formula. The Modern Language Journal 1956;40:433-41

Spaulding S. Two formulas for estimating the reading difficulty of Spanish. Educ Res Bull 1951; 30:117-24

Srinviasan L. Perspectives on nonformal adult learning. World Education, 1977. Functional Education for Individual, Community, and National Development. To order, contact: CBIS/EDRS, 7420 Fullerton Road, Suite 110, Springfield, VA 22153-2852; 1-800-443-3742 (publication no. ED329218)

US Department of Agriculture, Food and Nutrition Service, Office of Information. Guidelines: Writing for adults with limited reading skills. 1988. To order, contact: The Office of Information, 3101 Park Center Drive, Alexandria, VA 22302; (703) 305-2276

US Department of Health and Human Services. Center for Substance Abuse Prevention (CSAP). Numerous materials are available through CSAP's Clearinghouse. Especially relevant materials are: You Can Prepare Easy-To-Read Materials and You Can Use Communications Principles to Create Culturally Sensitive and Effective Materials. To order, contact: CSAP's National Clearinghouse for Alcohol and Drug Information, P.O. Box 2345, Rockville, MD 20847-2345; 1-800-729-6686 (1-800-SAY-NO-TO-DRUGS)

U.S. Department of Health and Human Services. Guide to preparing easy-to-read materials. Bethesda MD, National Heart, Lung, and Blood Institute, Smoking Education Program, 1989. To order, contact the NHLBI Information Center at (301) 251-1222.

US Department of Health and Human Services. Making health communication programs work: A planner's guide. Bethesda, MD: National Cancer Institute, Office of Cancer Communications,1989. To order, contact: NCl's Cancer Information Service at 1-800-4-CANCER

Venezky RL Kaestle CF, Sum AM. The subtle danger: Reflections on the literacy abilities of America's young adults. Princeton, NJ: Center for the Assessment of Educational Progress, Educational Testing Service, 1987

Walker A. Teaching the illiterate patient. J Enterostomal Ther 1987;14(2):83-6

Werner D. Where there is no doctor. A village health care handbook. 1982. To order, contact: The Hesperian Foundation, P.O. Box 1692, Palo Alto, CA 94302

White JV. Graphic design for the electronic age: The manual for traditional and desk top publishing. El Segundo, CA: Serif Publications: Xerox Corporation, 1988

Wileman R. Exercises in visual thinking. New York: Hastings House, 1980

Zimmerman M, Newton N, Frumin L, Wittet S. Developing health and family planning print materials for low-literate audiences: A guide. Washington, D.C.: Program for Appropriate Technology in Health,1989. To order, contact: PATH, 1990 M Street, N.W., Suite 700, Washington, D.C. 20036; (202) 822-0033



Software for
Determining
Reading Levels


Note to Reader: Each of these software programs should be available from your local computer store. Mention of these products does not constitute an endorsement by the National Cancer Institute.

Grammatik
Checks grammar and punctuation; assesses readability via sentence length, paragraph length, word length, use of technical terms, and use of passive voice. Can flag each "error" for review and correction.
To order, call local software store.


Readability Calculations
Analyzes a document's readability level using nine different readability formulas. To order, contact: Micropower and Light Company, Readability Calculations, 8814 Sanshire Avenue, Dallas, TX 75231; (214) 553-0105

Appendix 5: Organizational Resources


Academy for Educational Development, 1255 23rd Street, N. W., Washington, D.C. 20037; (202) 884-8700.
Works with organizations to develop basic health education programs. Many efforts are international in scope.

ACTION, Director,1100 Vermont Avenue, N.W., Washington, D.C. 20525; (202) 606-4853. Supports more than 450 retired senior volunteers and more than 200 VISTA volunteers serving in adult literacy programs. Offers grants for literacy volunteer activities.

Assault on Illiteracy Program (AOIP), Executive Director, 231 West 29th Street, Room 1205, New York, NY 10001; (212) 967-4008. Nonprofit organization that targets out-of-school blacks of all ages with literacy needs. Publishes the ADVANCER, a weekly supplement distributed with black-owned community newspapers.

Association for Community-Based Education, 1805 Florida Avenue, N.W., Washington, D.C. 20009; (202) 462-6333. Provides information and support to member groups that operate outside the public education establishment. Technical assistance for community and organizational assessment is available. Has research, training, and informational materials useful in community-based settings.

Barbara Bush Foundation for Family Literacy, 1002 Wisconsin Avenue, N.W., Washington, D.C. 20007; (202) 338-2006. The Barbara Bush Foundation seeks to identify effective literacy programs, provide and send money for community planning of interagency family literacy programs; award grants to establish intergenerational literacy efforts; support training and professional development for teachers; encourage recognition of volunteers, educators, students, and effective literacy programs.

Center for Substance Abuse Prevention (CSAP) Communications Team, 7200 Wisconsin Avenue, Suite 500, Bethesda, MD 20814-4820; (301) 951-3277. A service of the Center for Substance Abuse Prevention, the CSAP Communications Team assists community-based and other programs that are working to prevent alcohol and other drug problems.

Clearinghouse on Adult Education, U.S. Department of Education, Division of Adult Education, 330 C Street, S.W., Washington, D.C. 20202-7240, Attention: Education Program Specialist; (202) 205-8270. The Clearinghouse links the adult education community with existing resources in adult education, provides information on request, provides referral services, issues publications, and refers inquiries to appropriate information sources. A bibliography that includes a section on volunteerism is available free of charge.

Contact Literacy Center, P.O. Box 81826, Lincoln, NE 68501-1826; (800) 228-8813. The hotline takes telephone calls from potential volunteer tutors nationwide and from individuals interested in enrolling in literacy programs and refers them to existing adult education programs in their communities. It operates from 6:00 a.m. to midnight, Monday through Saturday, and noon to midnight on Sundays. Spanish speakers are available.

Education Programs Associates, Suite 40, One West Campbell, Campbell, CA 95008; (408) 374-3720. Develops low to moderate reading level materials on maternal and child health, nutrition, family planning and other health-related issues. Also develops Spanish-language materials. Conducts training for health professionals on developing, selecting, and evaluating low-literacy materials and on how to identify low-literate learners.

ERIC Clearinghouse on Adult, Career, and Vocational Education, The Center on Education and Training for Employment, 1900 Kenny Road, Columbus, OH 43210-1090; (800) 848-4815. The clearinghouse provides information resources on all aspects of adult education, including adult literacy, workplace literacy, and Adult Basic Education. A list of all materials that can be ordered is available free of charge.

Health Promotion Council of Southeastern Pennsylvania. 311 South Juniper Street,Suite 308,Philadelphia,PA 19107;(215)546-1276. Develops and distributes literacy materials geared towards African-American and Latino groups. Provides seminars and workshops for health professionals on health education for nonreaders.

Institute of Lifetime Learning, American Association of Retired Persons, Program Department, 601 E Street, N.W., Washington, D.C. 20049; (202) 434-2277. The Institute, the continuing education service of AARP, works to reduce illiteracy in America.

Institute for the Study of Adult Literacy, Pennsylvania State University, Suite 209, 204 Calder Way, University Park, PA 16801-4756; (814) 863-3777. Conducts research into adult literacy; develops videotapes, PSAs and computer-assisted instruction programs to teach literacy and basic skills.

Interactive Knowledge. P.O. Box 560865, Charlotte, NC 28256; (704) 344-0055. Interactive Knowledge markets two adult literacy coursework products to education and industry: The READY Course and the New Reader Bookstore. Interactive Knowledge is also involved in the development of workplace literacy applications using new instructional technologies such as CD-ROM, digital audio, and digital video.


International Diabetes Center, 3800 Park Nicollet Boulevard, Minneapolis, MN 55416; (888) 637-2675.
The International Diabetes Center has developed a low-literacy series to help diabetes educators teach people who cannot read well.

Laubach Literacy Action, U.S. Program of Laubach Literacy International, 1320 Jamesville Avenue, P.O. Box 131, Syracuse, NY 13210; (315) 422-9121. A national nonprofit organization, Laubach Literacy tutors adults reading below the sixth-grade level.

Literacy and Health Promotion Project. College of Osteopathic Medicine, University of New England, Hills Beach Road, Biddeford, Maine, 04005-9599; (207) 283-0171. Project trains health professionals regarding low-literacy materials development and communication issues.

Literacy Research Center, Daniel A. Wagner, Director, Graduate School of Education, University of Pennsylvania, 3700 Walnut Street, Philadelphia, PA 19104. Initiates and conducts literacy research projects and provides community service through a variety of consulting and collaborative activities. Publishes the Literacy Research Newsletter.

Literacy Volunteers of America, 5795 Widewaters Parkway, Syracuse, NY 13214-1846; (315) 445-8000. A national, nonprofit organization that combats illiteracy through a network of community literacy volunteer programs.

National Coalition for Literacy, c/o American Library Association, 50 East Huron Street, Chicago, IL 60611; (312) 944-6780. Please note: the "parent" organization for the coalition changes every few years.

National Council on the Aging, Inc., 409 3rd Street, S.W., Suite 200, Washington, D.C. 20024; (202) 479-1200. Provides publications on adult literacy.

National Institute for Literacy, 800 Connecticut Avenue, N.W., Suite 200, Washington, D.C. 20206-1560; (202) 632-1500. Serves as a national focal point for research, dissemination, interagency policy development, and program evaluation on literacy. Also seeks to improve access to and enhance effectiveness of literacy and basic skills programs.

Patient Learning Associates, Inc., Four Chilham Court, Potomac, MD 20854; (301) 340-9894. Conducts workshops for health professionals on working with and better understanding low-literate learners. Offers specific guidance in the development of low- literacy print, audio, and visual materials.

Project Literacy U.S., 4802 Fifth Avenue, Pittsburgh, PA 15213; (412) 622-1492. PLUS promotes adult literacy. A joint project of the Public Broadcasting Service and the American Broadcasting Corporation, PLUS uses media to increase awareness of literacy issues and to recruit individuals into literacy training programs.

Program for Appropriate Technology in Health, 1990 M Street, N.W., Suite 700, Washington, D.C. 20036; (202) 822-0033. PATH develops low-literacy materials and trains health professionals and lay educators to develop such materials in the U.S. and abroad.


Simply Put, 427 Dock Road Cedar Run, NJ 08092; (609) 494-4880.
Revises and clarifies instructions for consumer products, health care guidelines, and patient regimens. Conducts workshops on patient/health practitioner interactions-simple principles on how to enhance the learning process.


Women of the Evangelical Lutheran Church in America, 8765 West Higgins Road, Chicago, lL 60637; (312) 380-2736.
The volunteer aides program is the Nation's largest ecumenical church-sponsored literacy program.