Claremont McKenna College

APPLYING THE SCIENCE OF LEARNING

Critical Questions

We understand that asking the right questions is at least as important (arguably more so) than finding the right answers. What problems are we trying to solve, and how should we go about framing a question that is solvable? The way the questions are asked will guide the answers that we accept and will tell us much about the assumptions that we make about higher education. Many people agree that higher education needs to be fixed, but we can’t agree on what it is that is broken.

In preparing for the retreat, invited participants were asked to think about the following questions. One of our stellar graduate students (Art Zavala and Kathy Hayes) phoned each participant and tape-recorded their answers.

  • What are the findings or theories from your area of expertise that we could apply to higher education? (Feel free to define your area of expertise any way you want. Some examples of what we are thinking about include research on the transfer of training, understanding how people work in groups, encouraging creative or analytic thinking, cultural or motivational factors, learning with multimedia, and maintenance of attention.)
  • What are the (most important) unsolved problems? What should be included in an agenda for research?
  • What prototypes can you point us toward where principles from the science of learning are already being applied (e.g., activities, courses, fields of study, degree programs, or entire systems)?
  • What are the major problems with or barriers to redesigning higher education? Do you have any ideas for overcoming them?
  • What additional questions should we be asking?
  • What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

Here are summaries of the responses that they provided and they questions they thought we should be asking. Additional responses will be posted to this site as they become available.

Ralph Wolff

Area of Expertise

  • Leading assessment of student learning for all WASC accrediting schools
  • Action Research or educational effectiveness
  • Executive director of an accreditation association

What are the (most important) unsolved problems? What should be included in an agenda for research?

We do not have a common definition of learning. There are implicit definitions of learning. Our definition of learning grows out of the industrial model and the Newtonian physics model, in which learning is about mastery of facts. I have read research and I have found, consistent with what my observations are, that we confuse learning with mastery of fact. Faculty are also divided about what learning is and about whether learning can be measured or evaluated effectively. So it is necessary that we continue and enrich the conversation about learning and that our discussions are grounded on research. Students should also not be left out of this. We need to have faculty and students in learning centered conversations and discuss what is learning and what are the methods by which we acquire learning

The change in the demography of American higher education has forced us to really reassess what are the purposes and what are the indicators of effectiveness in our institutions. Higher education is no longer a privilege; it is perceived to be an absolute economic necessity by policy makers for continued economic growth and by students and parents to be the only ticket to success in the American culture. As a consequence the role of higher education has taken a huge significance and it has taken a vital role to the future success of the American dream. It has become an economic one, but nonetheless we now have a diversification of the educational system (e.g., typical students work 20 hours a week, the average age of the students in the CSU system is 28 instead of 21, typical student can be married with children). So we need new indicators of success and effectiveness. The only ones we are relying upon now are through put statistics—how many graduate in six years. But we do not have good qualitative measures. Most people would agree that the SAT and ACT are only good predictors of the first year of success and are really not intelligence tests, although they are viewed that way. So we need indicators of academic quality because we can no longer rely on the SAT and entry level statistics (GPA). There is an increasing need to know whether graduates will be able to do or know

The second point is that we are also at a critical time with the information revolution and the explosion of what there is to know and thus there is a desperate need to build connections to redefine what learning is. The model that most faculty were raised in, myself included, is that there are a set of facts that need to be mastered and each course is separate from one another. What students need is a different model of learning and knowing. A much different model that is generative, contextual, that is interrelated, and that truly understands the changing character of knowing. The foundations of every major discipline are rapidly changing, although there are some basics that are constant. Students need to be prepared for this change and this notion of fact mastery has got to give way. So we really need to rethink what learning and knowing is and whether curricula are actually structured for student learning.

We are also trained to teach and to tell, but not to induce learning. Faculty were trained to profess, but not to think about learning. Grant Wiggins tells this story that Michael Jordan didn’t become a great basketball player because he scored an A on it. Wiggins terms this feedback-based learning. True learning or mastery occurs through constant practices. A pianist does not become a concert pianist or a person does not get a black belt by reading about it, it is practiced and there is ton of feedback that is provided to them. We have a post-hoc assessment model of evaluating learning and teaching, but not a feedback and confidence-building model.

What are the major problems with or barriers to redesigning higher education? Do you have any ideas for overcoming them?

On major barrier is that there is not agreement that higher education needs to be redesigned. A lot of people you have in higher education are quite satisfied with it. We are the victims of our own success. We believe that we are the best system of higher education in the world. We are the richest, but I’m not sure we are the best anymore. I have been in other countries and met with enough other people to actually know that they don’t hold that view. We have the richest system, which means that for graduate study the Chinese and others want to send people here because of our research facilities. But at the undergraduate level that’s questionable.

Second, I think there is a lack of a willingness to experiment, particularly in the western part of the United States. There is an established paradigm that the only knowing is scientifically validated knowing. The only true work is that which is scientifically verifiable and within that only research universities can produce good science and that only quality comes from that or the elite institutions. There is just such a higherarchy and little experimentation.

What prototypes can you point us toward where principles from the science of learning are already being applied (e.g., activities, courses, fields of study, degree programs, or entire systems)?

There is a tremendous amount of work in teaching writing and it reflects feedback-based learning. In music and in art it is more of a feedback-based system, in which the criteria are as openly discussed as the product. But traditionally universities as a whole do not adopt a feedback-based learning system in terms of curriculum design and evaluation. Sure there are prototypes, there is Evergreen State and Fairhaven College of Western Washington, so there are a lot of those who take education in a more holistic way. I just saw that Washington state is trying to develop a whole community college based on integrative learning.

What additional questions should we be asking and what do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

Why would we even care about learning? Do we care? And if we do, then are we satisfied that what we are doing is sufficient. If the answer is we don’t care or that it is not our job to care and that the faculty are doing a good job and things are working well, then I don’t think what we produce at this meeting will create change. So the real question is are we satisfied with the level of learning? I would have to say that I do not feel satisfied with the level of learning or engagement, nor do I feel that faculty consistently employ methodology in their relationships with one another, let alone with the students, that are geared to stimulate the kinds of student learning and mastery that we are trying to accomplish in large enough numbers. We do produce a lot of wonderful graduates, but there are a lot of people who leave college feeling burnt out and devastated. The second issue is do we have a common basis to even talk about what even learning is. Thirdly, is there a protocol by which we can discuss and evaluate whether learning is effective? My biggest fear is that the only protocol we have is the post-hoc model called the test. Another issue is how do we make learning as interesting as cancer research. I think it is. How do we enlist faculty to make learning as energizing, interesting, and researchable rather then the byproduct of professing?

(Back to top)

Carol Tomlinson-Keasey

Some of the Challenges Facing Higher Education

  • Increased need for higher education. More people are attending college than in the past. In most of the 20th century people could survive without a college education, but as we enter a new century there is an increased need for people to attend college.
  • Changing Demographics. The increase in population combined with the first point that more people will need to attend college only exacerbates the problem.
  • Increasing Costs. In California we have been lucky that our public institutions have not faced the same increases as other institutions across the nation. Still approximately 15% of a family’s income goes towards education for a child that is of college age, whereas just a few years ago it was only 9%.
  • Lifelong Learners. We need to be educating people much longer and throughout their lives.

When these factors are added together it becomes apparent that higher education is not going to meet these challenges using its current model. This is where the use of technology and looking at different ways to approach the problem become incredibly relevant.

What are the (most important) unsolved problems? What should be included in an agenda for research?

  • On an academic scale our institutions are divided into departments and colleges that do not easily morph into new departments or colleges to reflect the way that the fields are growing. All of the funding and resources go to the departments and colleges not to the new interdisciplinary areas that people are excited about.
  • We need to know more about the way that students learn and especially how they learn via the new technology.

What prototypes can you point us toward where principles from the science of learning are already being applied (e.g., activities, courses, fields of study, degree programs, or entire systems)?

  • Rensalears' model for studio courses.
  • Also,interested in looking at open university options.

What are the major problems with or barriers to redesigning higher education?

  • We don’t change quickly.
  • Our institutions are designed to support a model that is time and place bound.

Do you have any ideas for overcoming them?

  • I think it was Sir John Daniel that said, “Technology is the answer. What is the question?”
  • There are many ways that technology will be helpful and that it can be used to help us overcome some of the barriers that we are not able to overcome as institutions.
  • This is not to oversimplify the problems and to say that technology will be able to solve all of the problems, but it will definitely be a solid tool with which to begin.

What additional questions should we be asking?

  • There are many levels on which to address each of the questions that have been asked.
  • Each question can be answered globally, for the U.S., for California, or for individual campuses.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

  • Outline the problem so that people are drawn away from their day-to-day character and begin to see that what they do really does impact education over the next 5 to 10 years.
  • Show people reasonable solutions.

(Back to top)

Sharon Riedel

(Back to top)

Anne Petersen

What are the findings or theories from your area of expertise that we could apply to higher education?

I have done research on cognitive functioning  especially gender differences in spatial ability.  I have a couple of books and probably a couple dozen research articles on the topic.  I've especially been interested in how biological and social factors influence cognitive functioning over the first 25 years of life.  This relates to higher education in understanding different learning styles of men and women (among other groups) in higher education.  For example, there is evidence that experiences  such as through sports  affect cognitive performance and probably brain development.

From a totally different part of my professional experience  that in university administration,  I've learned a lot about what affects student learning.  While at Penn State University, I was a member of a group examining undergraduate education  what practices enhance learning and what dampen it. For example, research demonstrates the minimal effectiveness (in terms of retention of information) of traditional lectures whereas more interactional teaching approaches have better learning outcomes.  Another memorable finding in this review of the learning research was that young women who drop out of science and technology majors do not do so because they are doing poorly in classes;  rather, they drop out because they find the classes boring (repetitious, requiring memorization and rote learning).

While at the National Science Foundation, we worked especially on efforts to emphasize hands on learning of science and technology  at all educational levels.  This approach has been effective at engaging young people in the science and math and motivating them to continue their studies in this area.  Higher Ed was the most difficult level at which to implement change.

Currently, I chair a Board at the National Academy of Sciences on Behavioral, Cognitive, and Sensory Sciences.  One of our committees examined the research on The Aging Mind (also the title of the resulting volume).  Among the fascinating conclusions of this inquiry (whose focus was to propose a research agenda for the National Institute of Aging in the area of cognition) was the effectiveness of a compensatory strategy to maintain performance competence.  For example, people could continue to perform cognitive tasks by using various ways to enhance memory, or continuing physical performance by reducing the number of things being done at once.  Compensatory strategies would seem useful to consider at various ages to deal with weaknesses in skills.  Similarly, a group examining learning at the other end of life concluded that activity based learning was much more likely to be retained.

Another inference arising from many professional experiences is the high salience of interdisciplinary learning for application in the real world.  Higher education has yet to implement interdisciplinary learning beyond various exploratory efforts;  most are not integrated into usual practice despite their appeal to students and relevance for most kinds of work.

What are the (most important) unsolved problems? What should be included in an agenda for research?

  1. Stereotyping of learners and its effects on performance over the short and long terms

    We know (from the research of Steele and others) that stereotyping exists, and that it has powerful effects in the near term. We don't know how pervasive these practices are, and we don't know the long term effects.

  2. Effective ways of changing teaching practice to increase learning of students

    We know about practices that increase learning.  Achieving effective implementation of these is less well known.

What prototypes can you point us toward where principles from the science of learning are already being applied (e.g., activities, courses, fields of study, degree programs, or entire systems)?

Many prototypes exist of most any important practice.  Challenge is to get these used more systematically.

What are the major problems with or barriers to redesigning higher education?

Entropy.

I suspect that change will require both top down and bottom up approaches.  It will require great patience and tremendous persistence.  Everyone needs to get on board with desired change and persist until it is fully integrated into practice.  It may take a generation.

What additional questions should we be asking?

I would be interested in knowing what faculty feel about the need to change.  I know that most administrators have been there for about a decade know.  Initially many felt that the challenge was a hopeless one.  More recently, things appear to be moving some.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

Both university leadership AND faculty need to want change and be supportive of the use of more effective approaches.

(Back to top)

Kaiping Peng

Area of Expertise

  • Cultural Psychologist.
  • Culture and Cognition
  • Study how culture-specific etiologies and theories and representations affect people’s thinking, reasoning, decision making, and emotion

What are the findings or theories from your area of expertise that we could apply to higher education?

Culture matters. There are cultural-specific etiologies and theories that affect people’s behavior, including the behaviors of learning something (as in an education setting). These issues have not been fully studied in the field of psychology. Sometimes they have been overlooked, not just internationally, but also in the US. We all know that many of the subjects in our studies, for many years, have traditionally been white male sophomores. We have not studied other ethnic groups to same extent, and we have not explored fully the impact of socioeconomic, racial, and other ethnic experiences and the ways in which these experiences may affect learning and education.

A lot of what we study in cultural psychology primarily focuses on definitions of culture and the formulation of cultural-specific theories., We have some of understanding of these issues, but we have not applied our understanding of cultural effects to higher education. We should be able to design better ways of teaching based on our knowledge of cultural influences on how people learn.

What are the (most important) unsolved problems? What should be included in an agenda for research?

Two kinds of problems:

  1. The problem of the independent variable. What is culture? Is it a purely cognitive, affective, or behavior? We do not know much about culture from a psychological perspective, even though we talk about it all the time. So the issue of what is culture is still not well understood.
  2. There are issues of the dependent variable as well. How broad are the cultural effects? How psychologically deep (in terms of mechanisms or processes) are these effects? We also talk a lot about multiculturalism, but mostly from a political perspective instead of a psychological one. We need to focus more on what are the cognitive benefits of multiculturalism.

What prototypes can you point us toward where principles from the science of learning are already being applied?

We know that there are differences in school performance by students from different ethnic groups. One example we all know is that Asian students perform better on math tests than any other ethnic group. There are a lot of theories about why they do so well in math. One possibility that has not been talked about much is that math is taught differently in Asian countries, which may be key to their success. Professor Howard Stevens at the University of Michigan provided one example of how teaching styles may affect performance. Dr. Stevens looked at an inner city school in Chicago with a predominantly Black student population. He found that the school did very well in a standard math test. He was very surprised to learn that the principal and the school in general emphasized what they called an “Asian way of teaching math.” So what is the Asian way of teaching math? One important component is that whenever you teach something, you do it in groups. New information is never taught solely with student-teacher interactions. An important part of the “Asian model” is an emphasis on students learning from each other. So they have a lot of group activities, and the activities are not simply presenting the answer as a group, but rather have the groups solve specific problems together. Also, when they solve the problems they do them in public. That is, they ask kids to solve the problem in front of the whole class, which can be very intimidating and humiliating, because if you do not know how to do it everyone will know. No American school or public school teacher would do this in the US because they believe that it would hurt the child’s self esteem. However, in Asian societies, teaching is always done this way. The public group method is based on the theory or folk belief that if you get learners excited or aroused, even by some humiliation, then they will be motivated to try and do their best. And even if learners don’t solve the problem, they remember the solution to the unsolved problem for quite sometime because they were aroused. When they applied the Asian way of teaching in that inner city Black school, the kids did very well. What I’m saying is that there are different practices from the ones we use in the US that might be helpful. We can look to other countries and other ethnic groups to understand alternative practices that we could be using.

What are the major problems with or barriers to redesigning higher education? Do you have any ideas for overcoming them?

Different beliefs about education may be a problem. People in the US have their own folk beliefs about education and about the intellectual development, which I sometimes call the Socrates view. The nature of the Socrates view about intellectual development can be categorized in this way:

  1. We emphasize questioning. That is, you have to question what people tell you.
  2. We also emphasize evaluation. That is, that you have to evaluate judgments about other peoples stories, theories, and results.
  3. We also emphasize self-generated knowledge.
  4. We believe that there is a single truth

These are ideas that are promoted and emphasized in the US. These ideas are not bad, but they are cultural-specific because people in other cultures do not share them. Asians views about education are more influenced by Confucius, which emphasize:

  1. Effort—effort for learning--you have to spend some time to learn something
  2. Respect—everybody has some knowledge that you don’t know.
  3. Dialectical—thinking from two different perspectives, contemplating the contradictions. For Westerners, the contradictions are error or mistake. For Asians, much of the intellectual and human life is about contradicting statements and they do not seem to feel the need to resolve contradictions

So I think those are the cultural specific views about intellectual development and cultural specific views about education. I think that the barriers to redesigning higher education are, in fact, cultural biases and cultural different beliefs.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

Reach out to the general public by way of the media. Inform them of our goals, and then present them with the major findings of the retreat. Publish articles that summarize the findings as well

(Back to top)

Vimla L. Patel

(Back to top)

John Newman

(Back to top)

Nora Newcombe

Area of Expertise

Development of spatial skills in children and adults, and how to facilitate skill development. Sex differences and other individual differences in higher level cognitive skills. Teaching cognitive science theories to educators.

What are the findings or theories from your area of expertise that we could apply to higher education?

Major concern is that there is a body of knowledge on such topics as analogical transfer, reading, and other issues concerning how people learn, that needs to be communicated to prospective teachers. Education majors need to have courses in cognitive science in the same way that engineering majors need physics and premed majors need biology.

What are the (most important) unsolved problems? What should be included in an agenda for research?

The major unsolved problem is a political one. There is a real need to convince colleges of education and also school districts that the knowledge provided by cognitive science is important for educators to have and to continually be trained in. How do we sell the achievements of cognitive science to those that could benefit from it, namely educators?

In looking at an agenda for research, we would want to look at the leverage points for achieving the transfer and wide scale acceptance of knowledge about cognitive science to educators. There is a need to fund demonstration projects that can exploit these leverage points and to evaluate their effectiveness. It should be a funding priority to have people explore technology transfer in cognitive science.

What prototypes can you point us toward where principles from the science of learning are already being applied (e.g., activities, courses, fields of study, degree programs, or entire systems)?

John Bransford and Marilyn Adams are good examples of people that have worked very effectively in developing cognitive science applications and actively interfacing with school districts. However, the work that they are doing is specific to types of problems and to age groups. So I would say that there are not any prototypes for the broad intertwining of cognitive science with colleges of education and schools in the way that I am suggesting.

What are the major problems with or barriers to redesigning higher education?

There are institutional barriers that are created by the current structuring of academic departments. Colleges of education do not traditionally employ cognitive scientists. The people who would need to teach the courses that I am envisioning would need to come from different departments and this creates turf issues

Do you have any ideas for overcoming them?

Nothing is as powerful in overcoming barriers as financial incentives. That is why there is a real need for funding demonstration projects that can illustrate the effectiveness of this type of training. Financial incentives to get colleges of education to work with cognitive psychology departments would be very helpful.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

It is typically the case that people will have personally rewarding and illuminating experiences at the conference or retreat and come up with interesting recommendations. However, as the outcomes are turned into a report written to the funding agency there is often no further monitoring or follow up. I think that it is important to see what could be done with Spencer Foundation and other funding agencies in terms of further monitoring and exchange after the writing of the report. It would be helpful to set up a format in which there was a meeting with Diane and others along with the funding agencies to create an exchange about the ideas that were generated. The follow up with funding agencies is key and it would be very beneficial to continue to develop funding contacts with other agencies so that the ideas generated by the retreat could be pursued.

(Back to top)

Jose Mestre

Area of Expertise

  • Learning in the sciences (particularly physics)
  • Study the following:
    • Problem solving
    • How to develop expertise
    • How do people organize knowledge in memory so that it is useful for retrieval and solving problems
  • How to apply research in the above areas to teach better, or more efficiently, or teach in ways that makes students learn the material at a deeper level

What are the (most important) unsolved problems? What should be included in an agenda for research?

  1. Lecturing to students in large classroom settings is not the most effective method of instruction. Findings that come from studies about how people learn indicate that whenever students are actively engaged in their learning, they learn better and faster. So, we need to find better ways of getting students to learn material than just lecturing in large classroom settings:
    • In a small class you can address students' needs more effectively
    • In large classes (50+ more students), which are typical of many introductory classes (in science in particular), lecturing is not effective. Other methods should be employed
      • Classroom Communications Systems. It is a version of the audience-polling device of “Who Wants to be a Millionaire.” Students are given questions to work on in groups and then submit answers that get displayed onto a histogram. The instructor then uses the histogram to direct the direction of the discussion in the class. The instructor is viewed more as a learning coach rather than a repeater of what is in the textbook. It has its benefits in the sense that the instructor can shape the instruction to meet the needs of the students…if students get it, then the instructor can move on, and if they don't then he/she needs to back track and figure out what is wrong and deal with it. Students seem to like it and they seem to get more engaged.
  2. Implementation—How do you get instructional techniques and implications of research into practice? Getting a charismatic instructor is not the answer. Charisma only goes so far; research evidence in the sciences suggests that students learn about the same amount regardless of who teaches them. How do you take things that work and scale them up so that other instructors are willing to adopt them—that’s a big problem. Two problems to implementation:

    1. To teach at the university level, you never have to take a course on teaching, pedagogy, or cognition (the only exception may be school of education professors). You become a teacher by getting a Ph.D. in some subject and it is assumed that content expertise is the only thing needed to teach that subject, and nowhere in between are you taught about instruction. In general, you are never taught about how to apply findings from research and learning to instruction in your discipline. We are essentially reproducing a failed system of instruction, and not because we don’t turn out good Ph.D.s, but we because we fail at teaching the general mass of students.
    2. People who have been teaching for a number of years think that they “have it down,” and feel that if they say it in the right way or make lectures a little more clear then students should “get it,” when, in fact, the issue is students have to be engaged in learning. Lecturing is really a boring way to go for students.

Telling people what to do to improve their teaching does not really help, even if they are willing to change. You may have to have an apprenticeship model where teachers observe or co-teach a subject or class with somebody else. Teachers might be able to adopt a new technique so that it meets their style and their students' needs with this learning model.

So, how do you train perspective Ph.D.s in all fields so that they know something about teaching and learning, and how do you get at the people who are teaching already to adopt new styles? These are big challenges.

What prototypes can you point us toward where principles from the science of learning are already being applied (e.g., activities, courses, fields of study, degree programs, or entire systems)?

  1. Books: How People Learn—has a number of chapters on teaching math, science, and history.
  2. Physics education research is becoming more accepted in physics departments. In our department and 10-12 other research departments around the nation, there are physics education research groups with doctoral students who do their dissertation on some aspect of physics education research. So you have some degree programs that are geared to help people with both the content and the teaching and learning of that content.

What are the major problems with or barriers to redesigning higher education? Do you have any ideas for overcoming them?

  1. Higher education has been very compartmentalized into departments, groups, and schools. And often times, those groups look down their noses at the other groups. For example, it is typical for scientists to look down their noses at schools of education. We need to break down barriers to solve the complex problems of education.
    • One example where barriers need to be broken is in the training of teachers. Every science department plays some role in the training of prospective teachers (at the high school level). So, if you are majoring in biology and you want to become a biology teacher, you take several biology classes. But, biologists teach those courses. You would have to go to the school of education to get the pedagogical methods course. However, based on what we know from the NRC's How People Learn report, the pedagogy and the content need to be taught together to develop "pedagogical content knowledge". Put a different way, just because I can teach physics well does not mean I could teach biology well.
    • So, although it is unlikely, there needs to be cross-disciplinary work 
  2. Telling people that they are doing something wrong is really not going to work.
    • One way the physics community was able to get faculty interested in finding effective ways of teaching large lecture courses was by developing a conceptual test in physics and encouraging faculty to administer it at the end of their courses. After students scored very poorly on the test, a test that most physics instructors considered "trivial" and thereby thought that their students would perform very well in it, the professors started to question what they were actually accomplishing by teaching in large lectures courses with the lecture method. What was clear was that students were able to solve "standard" problems, but they did not really understand the concepts. This opened up dialogue across many departments and faculty began to ask “How do I go about teaching in different ways.”
    • Once you have the attention of the faculty, it is important that they are given enough support, so that they are able to implement new techniques.

What additional questions should we be asking?

What are the goals and what are we (at this conference) trying to accomplish, and are they modest enough so that you can make some headway? Trying to reform higher education is a big job.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

Need administrators to be involved, either at the conference or at some point.

(Back to top)

Richard E. Mayer

Area of Expertise

Cognitive Psychologist, Cognition and Instruction

What are the findings or theories from your area of expertise that we could apply to higher education?

At the most general level I would say that we need a theory of learning, an understanding of how people learn if we want to design instruction that will help people learn. We need findings and theories that tell us how people learn. Essentially what we need is an educationally relevant theory of learning.

What are the (most important) unsolved problems? What should be included in an agenda for research?

There are very few solved problems, which leaves just about everything else as an unsolved problem. The question that I am most interested in is how can you teach in ways that promote transfer. What kinds of learning experiences allow people to take what they have learned and apply it in situations? Transfer is a classic issue in psychology and in education, but I think that it is really the fundamental unsolved problem. We really want to know how to teach for transfer. I think that we know how to teach facts and procedures, what we don’t know is how to teach higher-level concepts and strategies.

An agenda for research needs to include an educationally relevant theory of learning.

What prototypes can you point us toward where principles from the science of learning are already being applied?

We can all point to our own work. At this point in our research in learning we have decided that it does not make sense to have a general theory of learning and what we have are much more domain-specific models of learning. The theories that we are going to develop are going to have to be domain specific to a certain extent.

What are the major problems with or barriers to redesigning higher education?

One of the barriers is that we don’t have a good concept of what “good instruction” is and we don’t have good theories of what “good instruction” is.

Another barrier involves the use of technology in education. I have been interested in educational technology. I think that sometimes people confuse media and method. This is kind of a traditional issue in instructional technology. At the level of higher education people sometimes think that innovation means using technology. We know from many, many years of research on educational technology that technology does not create learning; it is instructional methods that cause learning. Just inserting technology into higher education is not really a viable solution. We need to find the instructional methods that help people learn and then see how technology can be used to serve that purpose. Many people take a technology-centered approach rather than a learning-centered approach.

Do you have any ideas for overcoming them?

To overcome these barriers we need more research. Research on what constitutes “good instruction” and research on how to use technology to improve learning.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

Until we know what constitutes “good teaching” I am not sure that we are ready for change. I am interested in what do we know about how people learn that would let us know what the change should be. Do we have a theory of learning that is relevant and what are its implications? I get nervous about people talking about change before we have really looked at what should be changed.

(Back to top)

Marsha Lovett

Area of Expertise

Cognitive Psychology; acquisition and transfer of problem solving skills

What are the findings or theories from your area of expertise that we could apply to higher education?

There are two empirical studies that I point to whose results can be used to improve instruction. In particular, these exemplify some of the simple things that can be done to enhance transfer and generalization of knowledge.

The first is by Cummins (1992: JEP:LMC) in the domain of algebra. She found that when subjects were given simple instructions to compare and contrast analogous problems, they were much more likely to induce a generalized problem schema (& transfer the problem solving skills). This finding helps to illustrate that some of the things we can do are very simple: participants benefited from the comparison task without even solving. It is often just a matter of asking questions in a way that prompts students to provide their own self-explanation or will allow some other processing that will allow students to generalize their results (see recent work by Chi that extends her work on the self-explanation effect).

The second study is by Catrambone (1998: JEP:G; 1996: JEP:LMC; 1994: Mem&Cog) regarding procedural skills and schemas. He was able to demonstrate that if you highlight the structure of problem solving steps in examples presented to subjects, the subjects are more likely to generalize the procedure to other problems. He found that this was even effective when subgoals were highlighted by being formatted into groups with arbitrary labels. His explanation for this was that highlighting the steps to students helps to lead to self-explanation.

Another important contribution to this is Singly and Anderson’s theory of transfer of skill (Singley & Anderson, 1989).

What are the (most important) unsolved problems?

The first one involves the research on epistemology and students’ beliefs about learning. There has been a considerable amount of work classifying students’ beliefs about knowledge or describing the impact of metacognitive beliefs on learning. Much less work has been done on understanding how students change their views on learning. Often this change has been thought to occur naturally as a result of maturation, but an unanswer question is: can we create interventions that will help to effect this change.

For example, students’ typical belief that learning is quick and problem solving is immediate is an obstacle to their learning: they think that if they can’t do something right away it can’t be done. The belief that authority on knowledge comes from “the teacher” or is “in the book” can also impede deeper learning. Again, the unsolved problem is: How can we help to change these beliefs?

The second issue involves the link between motivation and cognition. People tend to study either cognitive issues or motivational issues; an important problem is understanding (and dealin with) the interactive link between the two.

A final issue is how students should be scaffolded in their instruction (both computer-aided and other forms of instruction) and how and when this scaffolding should be removed. There is empirical work on this topic but not much theoretical development in this area. I believe that framing the empirical results in a theoretical structure (and extending them) would help to guide instructors in how to apply these results in new situations.

What should be included in an agenda for research?

In developmental psychology, we have gone from emphasizing the description of developmental states to actually describing the changes between states and the mechanisms for accomplishing such changes. In a similar way we could have people trying to understand the changes that occur in students’ epistemology (see first unsolved problem above), not just describing a student’s state of mind but describing the change process. Additionally, it is important to get people collaborating on different pieces of learning from a variety of perspectives, ranging from the cognitive to the motivational.

What prototypes can you point us toward where principles from the science of learning are already being applied (e.g., activities, courses, fields of study, degree programs, or entire systems)?

We are in the process of refining instructional design in an introductory statistics class at Carnegie Mellon. At each step of the redesign, we try to bring more of the principles of cognitive psychology and learning research into practice (cf. Lovett & Genovese, 1999 from The American Statistician; Lovett, in press: A collaborative convergence on studying reasoning processes: A case study in statistics. To appear in S. Carver & D. Klahr Cognition and Instruction: 25 Years of Progress. Mahway, NJ: Erlbaum). We put theory into practice when we select the materials for the course and design activities that engage students in learning. The last step of the process will be to incorporate a cognitive tutor to help students solve statistical problems based on their own level of understanding. This work is based on cognitive tutors that have been very successful at the high school level in algebra and geometry classes. The work of Ann Brown (Communities of Learning, I believe a relevant ref is from 1992 Journal of the Learning Sciences) is also a good example of interdisciplinary and collaborative work.

What are the major problems with or barriers to redesigning higher education?

Everyone is inventing their own wheel for solving instructional problems. People are not building enough on the work that has already been done and they fail to fully explore what is already out there, both in terms of what works and what does not work. One part of this problem is that the information is not readily available or easily accessible. The other piece of the problem is more of a cultural impediment in that people get invested in “name brand work.” This produces resistance for people to build on other people’s work.

Do you have any ideas for overcoming them?

The lack of information problem could be addressed with searchable databases or archives that could provide a place to reference work that is being done in applying cognitive and learning principles to higher education, and to document what works in different areas and what does not.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

After looking at some of the different perspectives on the most important problems, it would be helpful to find the commonalities in these problems and then to set out to have people attack these problems using shared resources. One success would be to get people communicating and looking for ways to get people together for longer-term collaborative projects. Herb Simon gave a speech at Carnegie Mellon on instruction and learning with a title something like, “Why isn’t teaching a team sport?” This title gets at the cultural issue of the solo aspect to instruction and instructional design. The statistics project we are working on is a very collaborative effort, with cognitive psychologists, instructional designers, statisticians, and software designers all working together to tackle this large problem. This requires extra effort (from developing a common language for communicating to figuring out how to meet with 5 other busy researchers for whom this is not a major project), but it would be great to get people to work on and maintain long-term collaborations.

(Back to top)

Joel R. Levin

  • Interested in improving the quality of evidence and the credibility of claims in educational research.
  • Focusing on the quality of research in education and psychology, especially, as it applies to policy implications.

Suggestions for Improving the Quality of Educational Research

The first suggestion would be to improve the design of educational research and to move toward more scientifically based models. It is important that we are not faddish in research questions and that we do not simply jump from one hot topic to another. There is a need to employ more systematic long-term research methodology. We need to be scientific in our method of inquiry—begin with a research hypothesis, test the hypothesis in small laboratory settings, and then move on to large scale, but very controlled settings (as in medicine, for example). We also, must not rely merely on demonstrations and uncontrolled studies, but instead, we should be guided by an approach where we would use multi-site testing in our research and use analyses that match the design of our research.

It is important that we begin to get away from testimony and opinion in education. Too many programs are put into place for political reasons and subjective opinions. New ideas hit the press and school districts implement these programs without really evaluating the effectiveness of these programs.

We need teams of people working together—interdisciplinary teams of people with strong scientific research backgrounds, along with people who are good in applied settings and in implementing programs. We need to entice people who know how to do scientifically sound research to work on field research and program evaluation, and we need to get the funding to do applied research. To answer questions about the effectiveness and outcomes of teaching strategies in large-scale classroom settings, as opposed to controlled laboratory studies of individual learners or short-term implementations of programs, there needs to be much bigger and better controlled studies than have previously been done. There is a need for classroom-trials studies (Levin & O’Donnell, 1999), research in classrooms that is modeled after clinical- trials research.

Additional Questions

  • What do we need to do to get people who are in positions to evaluate the evidence (funding agencies, policy makers, implementers of special programs) to be able to discriminate between differences in the quality of evidence? How do we build the understanding in people that not all evidence is equally credible?
  • How do you get both the people who evaluate programs and people outside of the education arenas who are not necessarily educated in scientific thinking to change the way that they think?

Programs for Students Who are Academically at Risk

  • Are there programs that can be used to help these students succeed?
  • Should we expect different educational outcomes for different students?
  • How do we address these concerns, and how do help such students succeed?
  • We know that with adequate resources and proper motivation and interest from faculty at-risk students will do better.
  • What are the critical components of successful retention programs for at-risk students?

Thoughts on Students’ Thinking and Learning

  • What is the relationship between memory and critical thinking?
  • Must information be memorized and readily accessible before it can be used in the service of thinking critically?
  • If so, would it be beneficial to help students learn underlying factual information in a shorter amount of time, so that they could spend more time on higher-level tasks?

(Back to top)

Alan M. Lesgold

What are the findings or theories from your area of expertise that we could apply to higher education?

Most of my work is in the area of developing technology so that students can “learn by doing.” As more opportunities for computer simulations and instructional design become available, we can provide more instruction that involves “learning by doing” as opposed to the traditional model of “learning by telling.” There are many areas where in the past we were only able to provide instruction through lecture or by having students memorize materials. In the past, when it was important, we would use simulation as our main method to provide the more active part of learning (i.e., clinical psychologists), however with technology we are now able to use active learning in a much greater number of areas. In the area of complex skills, we could be doing much more in terms of designing instruction that would help to bring together knowledge of skills with actually doing and working with these skills.

What are the (most important) unsolved problems?

The really hard problems lie in certain niche corners of the area of motivation. We have this whole area of distance learning that is developing, yet we don’t have much information about the variables that will help keep students motivated to continue through these classes. We don’t know much about what connections students need to interact with other students or with faculty in order to help facilitate their motivation to do the work on their own and to complete the class. When distance learning first began, prior to the current wave of computer classes, there was an initial track record that demonstrated a dropout rate of about 40-50%. Students would sign up, but would not finish the work for the class. While the situation does seem to have improved, there is still work to be done. For example, the University of Phoenix, which is a private institution where the primary focus has been on distance learning, still offers more hours of sit down class and primary instruction than they do network instruction. This is primarily because this is what we know how to do.

Motivation may also be a cover for skills of self-management and time-management. We need to tease apart these variables; therefore, it is important that cognitive psychologists along with motivational psychologists work on this problem.

We also need to develop a better understanding of the way that research and practice relate to each other. What are the ways that concepts ground themselves in experience? For education in general this is very important, but it is also relevant for higher education. We don’t have a solid understanding how different kinds of knowledge work together. How do different kinds of knowledge support one another and how do different kinds of knowledge interact with one another?

What should be included in an agenda for research?

Programs of multidisciplinary work connecting motivational efforts with metacognition efforts.

Factors that would help to contribute to success in the distance learning environment. Issues of time management are sometimes short-term problems, but some students can also have holistic problems with time-management. What are some of the ways that task analytic work could help? What could be done to create a clearer identification of the cognitive and motivational aspects that help a student succeed in a less socially manipulated area?

What prototypes can you point us toward where principles from the science of learning are already being applied (e.g., activities, courses, fields of study, degree programs, or entire systems)?

Intelligent Tutoring systems represent an application that is heavily grounded in primary theory. The work by John Anderson, Ken Koedinger, and Al Corbett in developing intelligent tutors is one example. The aim for these tutoring systems has been at the high school and middle school level, but they have also been used at community colleges.

There is also the use of intellegent tutoring systems for complex skills that has been used in the military and at the industry level.

Another example is work by Kurt VanLehn on tutors for physics and ways to use machine intellegence to help people do a better job of managing their study skills.

What are the major problems with or barriers to redesigning higher education?

A major barrier is the development of learning organizations and organizational research. Organizationally, universities have not learned how to build upon research on learning in designing their courses. The organization and structure of the academic world, tenure and the compartmentalization of academic areas, along with the mapping of academic freedom onto highly localized planning and decision making creates problems for implementing learning research. In looking at an agenda for educational research it is important that we include research on organizations.

Do you have any ideas for overcoming them?

Universities are adapting to these problems, but are doing so slowly. The number of tenured positions has been reduced, and there have been problems with the way non-tenured faculty are treated. The university benefits from the flexibility of non-tenure, but has not found a way to compensate its nontenured faculty properly. Change is taking place slowly; I don’t know if there is any way to speed this change process.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

The best that you can do is to put provocative ideas in front of the policy and decision makers. Along with the standard reports that come out, maybe the commissioning of work on some of the ideas/prototypes that come out of this meeting. No easy way to do this. Possibly presenting the information that comes from the retreat to a meeting of universities presidents and provosts.

Ideas for the Retreat

  • Setting aside time for presentations of the prototypes and demonstrations.
  • What data do we know about the students in distance learning classes?

(Back to top)

Daniel R. Ilgen

Area of Expertise

I/O psychologist; work motivation, teams and team decision-making

What are the findings or theories from your area of expertise that we could apply to higher education?

These are several areas that I/O psychologists have been traditionally involved in where the theories and findings could be applied to higher education.

  • Active Learning. There is a lot of work in training that works with active learning. Web-based training and life-long learning models have demonstrated the importance of treating the learner as an active participant in learning.
  • Decision-making.The use of choice models and issues related to the decisions that people make.
  • Teamwork. The idea that learning in teams is important, especially, since much of our behavior outside of the university is done in teams. We use teams in the educational setting; however, one of the implications of this is that we do not reward behavior in teams, we reward individual behavior—grades and diplomas are given to individuals not to teams. People are learning in teams and are learning about teamwork itself; this is a domain that I/0 psychology has worked with for a long time and has a lot to contribute.
  • Behavior between organizations. Education is moving from the sole property of the university to more of a joint venture between businesses, communities, universities and individuals. We don’t know much about how these areas will work together and what will happen as we cross these boundaries. What will be important in getting these various enterprises to work well together?
  • Competencies. Evaluating the competencies of individuals. I/O psychologists tend to work backwards by asking what is needed for the specific job and what is important to the nature of the job and then create the bundles of knowledge and competencies that would be expected. Education could benefit from a similar approach in attempting to understand how to bundle and cluster knowledge.
  • Task Motivation. Why do people stay on task? What kind of things keep students going? What things can be used to help with task motivation?

What are the (most important) unsolved problems? What should be included in an agenda for research?

Learning and the choice process. People are much more involved as active learners than we have previously thought or than we have looked at. The nature of learning puts much more burden on the person in terms of acquiring information. As an example, many people know how to use Microsoft Word, but the degree to which people learn the software often depends on their own level of motivation. Much more of the learning that we do is a process of our own willingness to go forward and the choices that we make in the learning process. We don’t know a lot about what keeps people motivated and keeps them pushing forward when they are left on their own to learn information.

Motivation. Businesses and universities are invading more and more on people’s personal lives. For example, businesses used to send employees to a retreat to get new information, now companies will often give employees a web site to go to and get the new information. At some point we have to ask how much we should be asking from people.

Team Work. How do we create the proper reward systems for work that is done in teams? As previously mentioned we don’t give degrees to teams, we give them to individuals. How can we structure a reward system for team and group work?

The first question is what level are we talking about in an agenda of research. We can do more research on web-based training, life-long learning, and training displays this research is at more of the micro level. At a different level we need to look at the psychological processes of being an active learner.

What prototypes can you point us toward where principles from the science of learning are already being applied?

Active Learning. In terms of active learning there is a lot of work going on in the area of web based training; we are learning what it takes for people to get the information and to be motivated to continue. Web based training is an example of active learning, but it is still very early in the development stages.

Michigan Virtual University. This is a consortium between the automotive industry, the junior colleges and the four-year colleges. Asking what is the way in which all of these organizations can work together to create a model. This is an area where organizational research will be important.

Collaborative work sites, also apply some of the principle of learning to the workplace.

What are the major problems with or barriers to redesigning higher education?

Higher Education is itself its own barrier. Universities are very conservative institutions, and there is a lot of resistance to change. The separation of the university into different academic departments means that we tend to compartmentalize our solutions to these problems. We stress interdisciplinary work, yet we continue to advance the level of knowledge in our disciplines so that people need to be more and more specialized. The more specialized that we are the harder it becomes to talk to anyone outside of our area.

Funding and these types of issues are also probably a concern, but not one that I am very familiar with.

Rapid change in technology results in things becoming outdated so quickly. By the time systems are purchased and people are trained in it; the technology is out of date.

Do you have any ideas for overcoming them?

Putting together interdisciplinary teams. Trying to get people to cross borders of disciplines. The constraints of the university are something that we just have to try to work around.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

We need a provocative report that could stir things up. I have always liked the National Research Counsel model. For example a book published by the Institute of Medicine titled “To Err is Human,” reported the number of medical errors in hospitals and other medical areas that lead to the loss of human lives. This study stirred things up. Although education does not have the same level of shock value—people dying as a result of mistakes—it would be great to have some report that could get people stirred up in a similar way. A group of experts that can get together and provide a provocative treatment of the problem is a great start to effecting change.

(Back to top)

Earl Hunt

What are the findings or theories from your area of expertise that we could apply to higher education?

The importance of building knowledge on knowledge. It is a fact that different people have different beliefs, even students that have passed through the same class. The diversity of beliefs creates a situation where individualizing instruction so that it is geared to address students’ different beliefs can be much more effective than the standard “stand and deliver” lecture. We now have the technological means to create an individualization of instruction. We have an opportunity to react to the students’ individual state of knowledge, as opposed to just lecturing to the group norm.

Differences in states of knowledge are qualitative differences, rather than just quantitative. This is not to say that all people’s knowledge is equal, but rather to say that people have different approaches to looking at a problem that are not necessarily right or wrong. Knowing how a person is approaching the problem and then tailoring instruction to fit the different approach that a person is taking to the problem will greatly improve the outcome of the instruction. Good instruction should engage a student and identify a student’s individual beliefs. This comes right out of schema theory and any theory on the organization of knowledge.

Use of Technology in the Classroom. We have designed a diagnostic project that can write diagnostic assessment programs. We can have continual self-assessment programs that can monitor the understanding of the students, and we can put these on the web. Instructors can get weekly reports, students can discuss their understandings with each other in chat rooms, and instructors can monitor these discussions and then modify the lectures and work with TAs to tailor the instruction to the students in the class. This is a great improvement over the pre-prepared lecture that does not adjust to the differences in individual classes. The horror of it was that you used to be considered a good instructor if all of your lectures were prepared before the start of the quarter; teaching this way is analogous to a ballistic missile, the lectures do not adapt to the new class of students nor to the new ideas that may develop through the course of the semester.

Additionally, academia is the only place where we have a system of assessment that Bob Inslay (?) once called “drop in from the sky assessment.” A typical course has a mid-term and a final. This form of assessment is terrible. The final is of no use as a teaching device; it is strictly a grading device. Most students don’t even come to pick up their finals. A mid-term is somewhat less useless as a guide for learning, but it is more often used as a device for students to monitor their progress. How often do instructors modify their instruction based on the results of the mid-term? Instead we should be doing continuous assessments, self-assessments that are built right into the instruction of the course. We have the ability to do this. Right now a student should not be told that on a 100 question multiple choice exam that they got 78 right and not given any further discussion as to what was right or wrong of why a problem was graded as right or wrong. Students will rarely go back to find out what they missed and why. I would really like to see education come to the point where there were no midterms and finals. I am not saying that we would get away from grading, but we could begin using more frequent self-assessments that provide feedback to the student and the instructor.

We should be assessing base knowledge consistently. In general, I think that we put too much effort in a single report. There are huge amounts of forgetting by students. In psychology, we know ways to help with this. One way is by building on levels of knowledge and by connecting the different pieces of knowledge. In mathematics, one course builds on the other—stabilizing your knowledge. In most of the other disciplines, educators take a very disjointed approach. Psychology, for example, is a very poorly designed curriculum. It jumps from one topic to the next—personality one day and history and systems the next. We do not build on the knowledge that the students are gaining.

What are the (most important) unsolved problems? What should be included in an agenda for research?

In order to see what I want done, it is necessary to have a substantial background of knowledge about how people think about this or that topic. What views are people likely to have? The diagnostic and assessment programs that I have discussed exist in areas such as physics and statistics, but they do not exist in many other areas because we have not done the research to see what it is that students’ know and don’t know in these areas.

We need to have psychologists and subject matter experts working together to determine the sorts of problems and misconceptions people have in these areas. The problem is that this is not exciting research, but we need to have an understanding of how student’s understand all of the academic subject areas in order to make continuous assessment work.

Another important area of research is in convincing people that learning is an active process, and at the same time making them realize that active learning does not always mean it will be fun. Some people believe that active learning means that you get to play a game, and game playing leads to learning. Americans tend to have the attitude that learning is either “easy” or “I can’t do it at all.” People often claim that they can’t do 8th grade algebra, something that is generally within most people’s range of abilities. The truth of the matter is that people often don’t want to put in the time and work that it would take to be able to do algebra. We also need to get rid of the attitude that it is not okay to ask questions or in any way to demonstrate that you don’t know something.

One of the problems in applying what psychologists know about the science of learning to education is related to the image of psychologists as clinicians. Psychologists are not seen as being relevant to the discussion of education.

I feel that some our biggest problems for redesign of higher education are not related to research, but are instead issues related to the institution. There is not a lot of motivation for professors to change their teaching methods. What is the incentive for a professor to do the things that I have suggested for a class?

Another problem is that students come with a certain set of expectations; it is almost an unwritten contract that the professors’ job is to tell them what will be on the final and that the students’ job is to memorize this information. The students have become accustomed to the memorization game, if you change the rules of the game by using methods such as continual assessment of learning; students are most likely going to be upset. It is going to make these students very nervous to have the rules changed. In a system where we rely on students’ evaluations of the professors, innovative teaching techniques might not be thought of well.

What prototypes can you point us toward where principles from the science of learning are already being applied?

David Maddigan’s work at University of Washington used exactly the types of methods that I have been talking about. I also point to Peter Falk’s work in New Mexico with computer-graded and latent indexing methods to improve students’ essays.

What are the major problems with or barriers to redesigning higher education? Do you have any ideas for overcoming them?

In addition to the many topics discussed previously, one of the biggest problems is the lack of incentive to change teaching methods. To illustrate this problem let’s look at the reward for an associate professor for being a good teacher versus the reward for the professor being a good researcher. Using my school, University of Washington as an example, an associate professor who has tenure can win a teaching award that amounts to about one month’s salary or the same professor could spend the time that would have taken on improving their teaching methods on applications for grants that could provide grant money of 2 month’s summer salary for the next 5 years. Why should that professor spend his time trying to get the long-shot teaching award instead of spending his time going after more l

(Back to top)

Keith J. Holyoak

(Back to top)

Robert Hoffman

Concept-Map

(Back to top)

Douglas J. Hermann

Area of Expertise

Cognition; cognitive technology

What are the findings or theories from your area of expertise that we could apply to higher education?

There are many, especially with the rapid advance of computer and other computer related devices. I have an especially strong interest in personal data-management devices. I have been interested in how palm-held devices can be used as a tool to help both time management and studying. There is also a large amount of technology on the web and different technologies that have come out of distance learning courses, such as graphic displays, that could be used in the classroom and also to advance personal learning.

What are the (most important) unsolved problems? What should be included in an agenda for research?

In the case of learning I think that the most important unsolved problem is how people can learn to focus their attention or their consciousness on a learning task. For example, when I talk with my students I tell them that I could give them new information about something that they are interested in, such as sports or music, and they could pick up the new information without having to take notes. They would be able to focus their attention on the new information in a way that they are not able to for a typical class lecture. I think that this is an area where we have a huge opportunity for success. If we can figure out ways to capture attention even when people are not necessarily trying to give it, we will have a real possibility to change learning.

Research is needed in how learning occurs in a broad framework. The environment of the educational setting must be included in the research in order to have a true sense of how learning takes place. For example, if someone is teaching in a school where there are many commuters, the effect of commuting on the ability to study and learn must be addressed. Each campus has an individual environment that creates different outcomes in learners. Often we look at learning in a strict situation of how long it takes a person to learn a particular amount of information; while this is needed research, it does not generalize well. We really need to look at the social context and setting of the school in order to understand how learning takes place.

What prototypes can you point us toward where principles from the science of learning are already being applied?

One example of a prototype is science classrooms that have computer communication between the students and the instructor. In these classrooms students have access to a computer keyboard that will allow them to respond to questions posed by the instructor. Students are able to respond electronically to simple “yes” or “no” questions or they can also respond to things such as a 5-point likert scale that rates the lecture from interesting to not interesting. This allows students to have more interaction in the lectures. Something that I have noticed with distance learning is that students are more willing to offer criticism over e-mail than they are in classroom situations. We now have the ability to use technology in a way that students can interact with the professor. This allows students an opportunity to express whether or not they understand material without the pressure of demonstrating their lack of knowledge to other students. Often when you ask students whether or not they understand a concept, it is only the top students that will give an answer.

What are the major problems with or barriers to redesigning higher education? Do you have any ideas for overcoming them?

I personally think that a very big problem is the relationship between basic and applied research. The idea in our society is that basic researchers generate some excellent ideas and applied researchers put these ideas in practice and then basic researchers come back and modify these ideas. Our students should be socialized into that process regardless of what field they enter. Students need to have an understanding of the give and take relationship between basic and applied researchers. The current state of science in America segregates basic researchers from applied researchers. I think that students see some of this and they do not have a good example to demonstrate how these two groups can work together. The productivity of American science is held back by that problem.

In order to overcome this barrier we need to create more communication between basic and applied researchers. By segregating the basic researchers from the applied researchers we perpetuate the problem. Conferences, such as this one, and other places where people can come together and share ideas from both sides of research are very important.

What additional questions should we be asking?

It might be good to look at questions about people that are not comfortable with technology. Is there something unique about the way these people interpret their environment?

Some other areas to consider relate to the use of distance learning. Should the same credit be applied for a class that is offered over the web as it is for the classroom? Is it possible that certain types of pedagogical activities are good over the web and that there are other pedagogical activities that are better for the classroom? To what extent are people able to learn through new modes of technology?

What techniques work to help get students to maintain attention? Electronic classrooms could go a long way towards this. If students could express their pleasure or displeasure about the lecture it would help students feel more interactive.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

Empower some practicums or small studies and then provide feedback from the study. I think that what is going to be important as well is for people to be able vocalize their resistance to the ideas of other people. There are lots of good people at this conference, but it is limited in time. People often do not want to be rude, but it is important that people are able to express their disagreement with ideas that they do not think will work. It is important to create an environment for people to feel able to open up and express their ideas.

(Back to top)

Diane F. Halpern

Concept-Map

Area of Expertise

  • Cognitive Psychology
  • Issues related to thinking

What are the findings or theories from your area of expertise that we could apply to higher education?

  • These are some of the pertinent questions that are addressed by cognitive psychology research in relation to higher education.
  • What are the things that we know that help to promote remembering beyond the time that someone is in school?
  • What is it that helps people to become better thinkers?
  • Is thinking something that happens in one specific domain of knowledge or are there general skills of thinking that would transfer among domains?
  • With all of these questions it is very important that we find good measures, so that we can accurately answer these questions in order to begin to apply this knowledge. Good measurement is essential if we are attempting to find out what works and what does not work.

What are the (most important) unsolved problems?

  • The unsolved problems are really about how to get transference. How do we get someone to spontaneously use the skills and knowledge that they have learned in class in other situations?
  • How do we keep people motivated and get students actively involved in the learning process?
  • How do we get people to accept that good learning is going to be hard work when all of the rewards and incentives are not set up to promote this?
  • How can we go about getting people to want to look at real research answers, and to not shy away from research answers that we do not like for fear of disappointment?

What should be included in an agenda for research?

  • List of what the most important questions are.
  • How do you do good applied research?
  • How do you get people to value evidence for what works and what doesn’t work, especially in the very politicized educational environment that we now have?

What prototypes can you point us toward where principles from the science of learning are already being applied (e.g., activities, courses, fields of study, degree programs, or entire systems)?

There do not appear to be that many instances of people really breaking down the walls of traditional teaching styles. One example of an alternative learning environment is the work that Donald Norman has been doing. He has created a more problem-based learning environment in online instruction. An example of the problem-based style is in the approach to teaching statistics. Students start with a data set and are then asked what are the interesting questions that relate to our business. Students are able to learn statistics through application as opposed to a traditional model.

What are the major problems with or barriers to redesigning higher education?

There are many barriers in the way of reform. Faculty unions are one barrier; unions establish a mindset of not doing extra work, and change requires a great deal of work. Changing the rules of the game will upset the current winners and losers in the system. There are also barriers as a result of state legislation and state agencies that are not willing to take risks. There are not enough faculty trained to make the changes, and it is also possible that both faculty and students do not understand the possible or potential benefits when the outcome is uncertain.

There are also systems of constraints, such as standardized tests like the GRE, and also, licensing exams. What happens to these requirements when the educational system begins to provide a different kind of knowledge than what these types of tests assess? The whole system needs to move at once.

Do you have any ideas for overcoming them?

Time and a few brave leaders willing to take the first big plunges; people who are willing to convince a whole network that there is a need for change and that this is what we should be doing. Possibly some very large national agency changing their learning structure, such as the military for example, that would have an impact on the whole society. Provided, of course, that this change could demonstrate a beneficial outcome.

What additional questions should we be asking?

We need to ask questions outside of cognitive science and extend our investigations to include social areas, groups, and group thinking. Also, questions about different means of presenting information. Another issue could be how to present these ideas to make them more palatable to the general public.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

We have to get someone to make a real commitment to take the good ideas that come from the retreat and put these ideas into action. There is a need to get someone or some groups to fund this process. We need an energetic leader to help support this and also a few really good demonstrations that could help to convert the general public.

(Back to top)

Milton D. Hakel

Concept-Map

Area of Expertise

  • I/O psychologist
  • Motivational factors and learning

What are the (most important) unsolved problems?

How can we stop wiping out our kids in our educational system? A specific example of this can be found in the rankings of our school children in math and science. American 4th grade students ranked 2nd in international comparisons of math and science while 8th grade students ranked around 35th. Is this merely a cohort effect or is there something that happens in the education system and/or the American culture that alters the motivational aspects of learning?

How can we encourage students to become engaged in the learning process? For example, undergraduates at my university (Bowling Green) are somewhat interested in learning, but are often more preoccupied with grades and ultimately with serving their time at the university in order to get to the next step and get a job.

What should be included in an agenda for research?

The first step would be creating specific starting points for research. The How People Learn book should be a good place to start. Hopefully, this conference will crystallize specific spots for the various areas of psychology to concentrate on in regard to why higher learning does not take place to the level that we would expect.

What prototypes can you point us toward where principles from the science of learning are already being applied (e.g., activities, courses, fields of study, degree programs, or entire systems)?

Alverno College in Milwaukee, Wisconsin is an excellent prototype of an entire system. This college applies assessment center methods to undergraduate education. Assessment center methods are derived from industry and military settings. Rather than interviewing a student or giving a multiple-choice test, you simulate situations and observe how the student performs in these situations. This method has been the basis for Alverno’s curriculum for over 25 years. I just reviewed Learning That Lasts: Integrating Learning, Development, and Performance in College and Beyond (Mentkowski and Associates, Jossey-Bass, 2000) for Psychological Science. The review is in press, and can be read at http://folios.bgsu.edu/Hakel.

Another prototype is a freshman seminar that I have been working on for the last four years. The class is officially called “Personal Development Seminar, but it is popularly known as “Springboard.” Twelve students are enrolled in each section, and each person has his or her own coach. Last fall we ran 26 sections, about 9% of the incoming first year cohort. The students are engaged in hands-on, active learning geared at developing communication, critical thinking, and leadership skills. Many activities are recorded on videotape, and coach and peer feedback are used heavily. Retention rates of those who have taken the class are higher than their cohort. Springboard shows that students are eager to learn; we need to find ways to overcome the barriers that inhibit learning among faculty and students.

What are the major problems with or barriers to redesigning higher education?

Curricular gridlock--universities are optimized for getting funding through enrollment. Due to the class sizes, lecture formats are often used exclusively and students are being exposed to knowledge, but are not engaged in applying that knowledge.

Other barriers include lots of inertia, and the large variety of stakeholders.

Do you have any ideas for overcoming them?

Having this conference is the first step. The fast sign-up of participants along with the excellence of their work makes me optimistic about very positive outcomes. This conference should help to pinpoint specific areas to begin pilot projects, as well as possible sources for funding these ideas. It is heartening to know that there are a lot of basic researchers out there that are interested in applying research.

People have sometimes said that if it is not broke, don’t fix it. However, there are more and more concerns about our education system and a lot of fears about being left behind as we enter into world globalization.

What additional questions should we be asking?

How can we create networks and connections that will help to continue the development of these ideas? What venues and to what audiences should we be addressing these concerns?

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

Some ways to do this include being very clear about what are messages are and to aim high. Reforming education is something that can only happen if someone decides to try it. It would be good to get at least half of the participants galvanized in some sense and working on these things either in their own labs and campuses or in teams working on demonstrations. This would be a good benchmark. As Karl Weick has argued it in an article in American Psychologist (Small Wins: Refefining the Scale of Social Problems, 1984, vol. 39, 40-49), there is importance in the psychology of small wins. Instead of waiting for all of the pieces to fall together, it is important to do things that can be done now in order to build confidence that will help lead to a major impact.

It will also be very beneficial to look at and display prototypes prior to and at the retreat.

(Back to top)

Arthur C. Graesser

Area of expertise

Discourse Processing

What are the findings or theories from your area of expertise that we could apply to higher education?

One of the big problems in education is the over-reliance on shallow knowledge. Students are often given key concepts and definitions of these concepts, but are not given a deeper level of understanding about these concepts or the ability to reason and problem solve. The knowledge students have is inert rather than active. The key question becomes what strategies can be applied that allow students to get to the higher level areas of thinking?

What are the (most important) unsolved problems?  What should be included in an agenda for research?

There are several theories from cognitive psychology and discourse processing that provide very salient solutions to problems of learning and of facilitating higher reasoning abilities. One of the big challenges is to get these theories to the education, government, and industry audiences. Another big problem is “edutainment,” educational forms of media that focus more on the entertainment facet rather than on pedagogy. We need to be able to mix the sizzle of “edutainment” with pedagogical understanding. Not enough research has been done on how people process animation or multimedia sources of information. The issue involves not only how this information is initially processed, but additionally, how people are able to use this information later on. There is a need for more longitudinal research with these new modes of presentation.

What prototypes can you point us toward where principles from the science of learning are already applied?

These are 3 examples of multimedia-based technologies.

  1. John Bransford and his interactive video series, i.e. Jasper and Beyond Jasper. Note: Little Planet is another more recent one, but check out his Vanderbilt site. These are videotapes of groups of people talking about how to solve a problem and managing the dialogue so that people are using it to solve problems.
  2. Intelligent Tutoring Systems- University of Pittsburgh. Ande’s and Atlas tutor (Hurt VanLehn) teach adults physics using conversational dialogues. Automated computer conversations aid students in their problem solving abilities. Also Alan Lesgold’s SHERLOCK system trains military personnel on equipment repair and maintenance.

    At Carnegie Melon, the PACT tutor by John Anderson and Kenneth Koedinger teach students algebra and geometry. His recent system has a natural language and conversational dialogue facility.
  3. Conversational agents- students have conversations with a computer; pedagogical agents talk to the learner. This will be helpful with lower literacy students; conversational presence will be helpful. James Lester’s system “Steve” has a virtual reality environment in which adults learn about operating mechanical and electronic equipment by interacting in VR. It is very conversational and gives hints. Also Art Graesser’s AutoTutor hold a conversation with a student in natural language as they learn about computer literacy; it has a talking head.

What are the major problems with or barriers to redesigning higher education?

The teacher workforce has not had access to multimedia and pedagogical strategies. Funding for teacher education on new types of teaching technologies is underwhelming. An estimate from the Wall Street Journal of what it would cost to update education with needed technologies is somewhere around $250 billion, with about half of that amount going towards educating teachers. State universities, in particular, are under-funded and do not have the most recent technologies. There needs to be a partnership between universities and corporate business to help bring this technology to the classroom. In addition, it is important that funding be provided not just for the technology, but also, for teacher training, instruction, and equipment and software maintenance.

There is a schism that exists in education between those who want to use science as a model versus those who want to use a non-scientific model (building more on a cultural enterprise). In talking about what strategies work, it is a challenge to convince the non-science people to adopt the principles that science suggests. Cognitive science has real strategies that work, but there is often real resistance to using them--both in the education systems and in outside agencies.

Do you have any ideas for overcoming them?

There needs to be greater emphasis on longitudinal research. One of the problems with psychology is the “one hour studies.” These do not provide accurate information about learning. Often the learning strategies that we are interested in take time to evoke, and it is expensive to do research that can evaluate these long-term strategies.

There is a real need to find a way get more sectors integrated; an attempt to bring big business, education, and government into the research that cognitive scientists are doing. There are not enough scientists to go around to report all of the work that they are doing.

What additional questions should we be asking?

How do we manage interdisciplinary research teams? There are some real difficulties with interdisciplinary research; it is often very hard to get people from different areas to work together without conflict. However, it is crucial that there be an interdisciplinary approach to questions of learning.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

Bring someone in from Office of Educational Research and also pivotal people in Washington. It might be beneficial to encourage someone from the learning marketing area, someone that is planning on entering the “edutainment” industry in a big way to become involved (perhaps Disney or Microsoft). Cognitive scientists have some very tangible solutions to learning problems and they need to find a way to market and sell these ideas.

(Back to top)

Don J. Foss

Area of Expertise

These days I have multiple hats; I am as much a higher education administrator (Dean of Arts and Science) as I am an experimental psychologist. In my administrative role, I care a lot about the progress of people learning, e.g., to write in freshman English or acquiring college math skills. I have thought some about how we can use the principles from cognitive science in a way that will help us to achieve these goals. Additionally, I am very interested in the teaching of foreign language. One thing that I would consider is hiring a cognitive engineer in a full time position to try to meet these goals, especially if it were part of an initiative with wider scope.

What are the findings or theories from your area of expertise that we could apply to higher education?

I am interested in the study of expertise and what it has to tell us about skill learning. I am fortunate because at Florida State there are people such as Anders Ericsson, for whom expertise is his area of expertise. I have tried to get myself “up to speed” on this area, as well as on the topic of transfer of training. I have had Robert Bjork come to Florida State to lecture on his findings and what he knows about this topic.

I am personally a psycholinguist and have thought some about applications of psycholinguist research to second language acquisition, but I have not had the time to develop this as yet.

What are the (most important) unsolved problems? What should be included in an agenda for research?

There are many, but one that is on my mind is the issue of retention and transfer of training. For example, if students take a course in basic math and then take a course in physics, they should be able to reach back and build on their basic knowledge, but often they are not able to do so. The same can be seen in upper division psychology classes where we assume that students will have an understanding of the elementary statistics course they took, and frequently they do not.

Additionally, both as a psycholinguist and as a Dean, I am interested in questions of skill acquisition and transfer in the writing process. Part of what we are supposed to be doing is giving people the skills to analyze complex issues and the abilities to write about them. It is an issue of teaching techniques that people can then use to generalize to other problems. This question of transfer is a classic problem, but one that I think still requires careful work.

To begin with, we need more research on the above topics. Another area for research concerns: what are useful applications of technology in the educational settings of today? Almost every educational institution has invested millions of dollars in technology, and I am not sure if the trading of capital for labor has paid off. We have done some work here at Florida State where we have begun to examine the use of computer laboratory time in our beginning math courses. We are looking at the effect of offsetting the discussion sessions with computer-based laboratory time and asking questions about our ability to increase students comprehension and, at the same time, reduce the time spent on the task. This is an area that needs more research.

Another topic has to do with motivation of students. How do you motivate students to devote time to the task?

What prototypes can you point us toward where principles from the science of learning are already being applied?

We have one here in our freshman math class at Florida State. I know that there are many others around, but this is the one I most familiar with. In some of these courses, we have substituted the recitation sections with computer-based problem solving sections. In these sections there are also people in the computer lab who are there to answer students’ questions. We have implemented this in a way that permits us do look at the results in a quasi-experimental way. We have not looked at the details of why this is working. One of my speculations is that this approach allows a much more individualized approach to the students’ problem solving. However, more research is needed before we can say much about why this is working.

What are the major problems with or barriers to redesigning higher education? Do you have any ideas for overcoming them?

I think that most people in most disciplines, including psychologists, have a slight disconnect between how they talk about these issues when they are wearing their research hats, and what they do in the classroom. Part of the problem is that we don’t have enough training for graduate students who will be teaching. There is a certain disdain for methodology in teaching. This may be diminishing in some places where they have developed training programs for graduate student teaching assistants. In speaking with teaching assistants in the math recitation sections here I have found that most do not think about the notion of analyzing what the students think and why they may have not been able to solve the problem. We need to train teaching assistants and one of the big problems involved with this is that we don’t having training of the trainers. It is very difficult to get these ideas to permeate through an institution.

What additional questions should we be asking?

Suppose that we are called in front of congress or in front of a judge and asked the question: what do you swear will work? What do we swear will work is the question that I think that we need to be asking. In this regard my target audience has been turned now to the training of graduate student teaching assistants. What training can we give them so that we know what they are doing is really affecting change in the students.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

We need to get confident about one or two things that we want to accomplish and then to talk seriously about how to institutionalize it. This will require some organization, such as NSF, to provide incentives for universities or other educational organizations to adopt the principles that people have come to agree on.

(Back to top)

Alan Feldman

Area of Expertise

My area of expertise is in teaching advanced placement psychology to inner city high school students, and getting them to learn to love psychology, and learning in general. I am certified as psychology, history and mathematics teacher and am experienced in teaching all three. Since I have taught at a local community college as an adjunct for seven years (mostly introductory and child development) I se the large discrepancies that exist between college and high school students in motivation and attitude toward learning. I believe that this has made me a better teacher.

What are the findings or theories from your area of expertise that we could apply to higher education?

My first suggestion in regards to this question deals with the university programs that train elementary through high school level teachers. I firmly believe (as does Diane Ravitch in Left Back) that teachers need far more content expertise. If there is one thing I am certain of in education; it is that knowing how to teach a subject comes more from knowing the material than any other single factor. Even among some of the college faculty that I have interacted with there seems to be a lack of joy in learning about their subject. I think that at all levels of education including higher levels there should always be time for discussing what current experts (scientists) in their respective fields are doing currently. I will never forget when a bright sixth grade boy in a class I was covering asked me if any scientists are alive! Yet, how many college freshmen in chemistry, history, psychology, biology-etc can name an important current scientist in the subject? Students get excitement from an instructor who talks about advances in their own areas of knowledge that are currently being discovered.

To develop transfer of training and to maintain attention, again I refer to the knowledge and excitement level of the teacher. Asking difficult challenging questions that create dissonance in students is an effective way to do both. I usually start each section with a comment or statement about a phenomenon in the world that is true but outside the student’s ability to explain. Each of these conundrums can be answered by selectively using the information in the next section of the text or notes. In other words I favor a guided constructivism.

Additionally for all students (especially urban students) at all levels teaching and assessing in a creative, analytic and practical fashion is important. The inner city student thrives on practical teaching strategies and tasks. This is validated by the research of Robert Sternberg of Yale in numerous studies. Students taught and assessed with all three approaches do better even if they are only tested with analytical questions, which is the norm.

What are the (most important) unsolved problems? What should be included in an agenda for research?

One important unsolved problem is the establishment of better teacher education training programs, including for college professors. Beginning teachers at all levels should be enrolled in a mentorship with an experienced colleague. I hope that some day local universities and colleges would supply links between their staff members and high school teachers to assist with content knowledge if needed, and what they want beginning college student to know. This also might encourage more students to attend that university.

Another area that is lacking is in the development of the effective use of heuristics in education. General heuristics are often not that helpful to student but specific content strategies are.

Teachers need to be shown what some scholars have called the microstructure of teaching-what behaviors they should specifically do to get an idea across effectively. How does one use scaffolding and protocol analysis in a particular subject area?

A third area that is problematic is he type of reading materials that are mandatory in introductory English courses in high school and college. I think that a major goal of teachers, in particular English teachers, is to develop in students a love of reading prior to forcing them to read great books of literature, which are often beyond the student’s current ability. For example, many high schools force students to read 5-7 Shakespeare plays in one year. College freshmen are forced to read books for which they have not developed the necessary prerequisite skills.

What prototypes can you point us toward where principles from the science of learning are already being applied?

Activities based on giving students analytic, practical and creative tasks are available, as is the information on developing ones own curriculum to establish such a model. The research and suggestions of Lawrence Steinberg et al, as described in his book Beyond The Classroom could greatly benefit the instruction of inner city high school and college students.

I am fond of the writings and research of Carol Dweck, Ellen Langer, John Bruer, Robert Sternberg, Peter Gray, John Bransford, Diane Halpern and Roger Schank. All have specific suggestions to improve education.

What are the major problems with or barriers to redesigning higher education? Do you have any ideas for overcoming them?

There is resistance to change in quarters where the individuals in charge often do not read or understand how to evaluate research. Every university should have a leading cognitive scientist be the assistant to the President. I am not kidding!

Due to a lack of funds many universities can not help students that need extra instruction. Many students want to do well, but often feel they need extra assistance especially in science and mathematics courses. The development of free web pages where students could at anytime have access to extra material to assist them with their course work. Establish programs where each university senior is mandated to tutor underclassman for a few hours per week possibly as a graduation requirement, or for some incentive.

What additional questions should we be asking?

How can the dissemination of information from the conference be made most accessible to those that can benefit from it?

How can the Internet best be used to improve instruction. Web based courses seem to be on the way in-what guidelines should be established?

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

Give specific task to conference members with time lines.

The recommendations can not be too global or vague.

Sending out information on certain effective teaching strategies with the instructions on how to do an in-service on the topic so that all professors or teachers can benefit.

Get something published in a number of venues with specific recommendations for the improvement of education-from Early Childhood Education to the Chronicle of Higher Education. Even having something in Psychology today that has a wider readership. Sending a copy of specific recommendations to every school of education in the United States.

Create a well-publicized web page with links to numerous content area specialists that list materials and specific teaching strategies with bibliographies.

(Back to top)

Howard T. Everson

What is your Area of Expertise and what are the findings or theories from your area that we could apply to higher education?

  • Educational Psychology
  • Cognitive Psychology
    • Metacognition, particularly around the issue of how to measure certain aspects of metacognition and what measurements look like and what the correlations look like with college level learning. For example, there is some work that we are doing in the area of metacognition and self-regulated learning that suggests that interventions early on (e.g., at the secondary school level) may help students achieve more in college.
  • Educational Measurement
    • Correlates of performance on academic cognitive abilities tests like the SAT and success in academic settings and the workplace.

What are the (most important) unsolved problems? What should be included in an agenda for research?

  1. Getting a better understanding of how expertise is developed and nurtured through instruction.
  2. Transfer of learning across domains. From a measurement perspective, trying to understand how to gauge student learning and achievement at the postsecondary level has been very difficult and elusive. Getting professionals and academics and others to agree on the forms of learning and academic achievement that are really valued at the secondary and post-secondary level.
  3. The cognitive framework for understanding learning styles and the interaction of learning styles with questions of student aptitudes and abilities to learn from different forms of instruction.
  4. We need theoretical help in understanding and characterizing learning within specific domains so that we can do a better job of providing diagnosis in those domain areas.

What prototypes can you point us toward where principles from the science of learning are already being applied?

We are making some progress with respect to understanding the frameworks for learning from instruction. We are getting a better sense of what role prior knowledge plays in learning and how to make learning more active and a constructive process.

We also seem to understand what factors have been motivating students to go on and develop their academic ability. The work on self-regulated and its effectiveness on learning have been very insightful.

The area of cognitive neurosciences has also advanced our knowledge of how our brain organizes information and how that can help us to understand individual differences, but also how to design learning environments to promote active learning. It is rather scattered and not well organized or unified in the sense that somebody can put their hands on, but that is part of the challenge to us.

What are the major problems with or barriers to redesigning higher education?

Faculty attitudes. One of the major barriers has to do with the domain specificity and control that faculty have over specific areas. Many people who are teaching at postsecondary level are not good teachers. They may understand their discipline very well or be expert in their disciplines, but that does not mean that they have the ability to organize the material, present it effectively, and design instruction in ways that maximize learning. A lot of what goes on in higher education classrooms is poor in the sense of having low quality instructional design.

Measure student learning. Many faculty have no understanding of how to measure student learning. They give examinations that are poorly constructed and they give tests that are not well calibrated or easily scored. Faculty cannot articulate to the students the kinds of standards or the knowledge, skills, and abilities that they are looking to measure and that they want students to measure. So it is not uncommon to go into a large lecture course in the first or second year of college and see poorly constructed multiple choice exams or short answer kinds of exams that place a heavy emphasis on short term memory processes, but don’t value or measure very well deep conceptual understanding on the part of the students.

Do you have any ideas for overcoming them?

Faculty need to be given exemplars of good instructional design looks like. We need to communicate to them the principles of effective instructional design. We need to share with them in a more systematic way theories of human learning so they begin to approach their tasks with a stronger theoretical framework of how humans learn.

What additional questions should we be asking?

We need to ask questions about how the various disciplines are organized cognitively and conceptually, so that we are asking about what contributes to the cognitive complexity within a discipline. If we had a better understanding of this we can be more prescriptive with respect to instructional design.

We have to ask questions about the role of collaborative learning activities and what they mean for instruction and assessment in postsecondary education.

We should ask questions about the effects of school reform initiatives at the secondary school level and what they imply for the kinds of teaching that takes place at the postsecondary school level. Many students in the future will emerge from secondary schools where the teaching and learning practices in those secondary schools will have been dramatically different than the teaching and learning practices they will encounter at the colleges and universities. They may have trouble adapting and succeeding as a result.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

We need develop a strategic plan that emerges from the retreat, where we have a set of research and implementation goals. I would like to see a strategic plan emerge that was broad based and emphasized collaboration with other groups and organizations interested in issues of student learning and instructional design.

We also need strategies for seeking funding, so that we do not loose momentum.

(Back to top)

Kevin Dunbar

Area of Expertise

  • Research on how scientists think and reason and how scientists train graduate students and post-docs. Over the past decade we have investigated the ways that scientists think “live” at lab meetings and how students in the labs are trained to conduct science. We have also gone back into our own lab and conducted controlled experiments on phenomena that we have identified in naturalistic settings. This methodology I have labeled InVivo/InVitro Cognition.

What are the findings or theories from your area of expertise that we could apply to higher education?

Analogy is a key process used to both educate scientists and to help scientists make discoveries. The classic findings in psychology are that people have great difficulty at using analogy; either in remembering analogs or using analogs to solve problems. Experimenters have shown that people tend to rely on superficial similarities between components of an analogy, rather than the underlying structural relations. In my research, we have found that analogy both based on superficial features and on deep structural features is frequently used. What is most important is the goal that the reasoner has. Superficial similarities are good at fixing minor problems, more structural analogies are good for formulating hypotheses, and really far analogies are good for explaining things to others. I have also found that analogy is a key to training scientists and that it is key to helping scientists make discoveries. Education must tailor the analogies to fit the particular goal. In addition, there are inherent dangers in making analogies, such as only seeing the similarities and not seeing the differences that can lead people astray. These problems with analogy also need to be confronted.

Distributed cognition. What happens to thinking and problem solving in the group process? What type of interactions lead people to make new discoveries and learn new things? The composition of the group is a key factor. Groups derived of people from the same background do not solve problems as easily as groups with people from different backgrounds with similar goals. One of the problems is that people from the same background all draw from the same base of knowledge, those with different backgrounds are able to pull in different types of knowledge.

If you want to use groups to learn, it is good to have groups with different backgrounds. The key, however, is that the groups must have a common goal. If group members have different goals, the group will not learn, and the results can be disastrous. I have studied groups with common goals and with different goals and found that the reasoning and problem solving strategies used by the groups differs widely. While collaborative learning is a hot topic in education, it is important to structure the groups in ways that will facilitate learning. Overlapping goals and knowledge is the key factor.

Training Methods. We have also been looking at how people plan experiments and have found that the classic training in experimental design (using control groups and experimental groups) is often too basic for the work that current scientists are doing. When we are training scientists, we are often doing so in very simple ways. Simple research designs are not consistent with what scientists actually do. For the most part, scientists are currently trained in an apprenticeship method. Students are expected to learn experimentation by being in the lab, as if through osmosis. This sort of training for future scientists is very much a hit or miss process. It can result in a loss of some very talented students that are just not given enough instruction or motivation to continue to become scientists. Thus, we are faced with the problem of preventing talented students from dropping out. This is a very large problem all across the educational spectrum.

Unexpected findings in research. Unexpected findings account for about 50-80% of the findings. There is little explicit training on how to do this and scientist are very vague and quasi religious about what they really do talking of the goddess Fortuna and flashes of insight. We have found that rather than being the victims of chance, scientists have well developed strategies for dealing with the unexpected. Professors tend to blame the method; post-docs and grad students tend to think that they have found something new. The types of analogies used and distributed reasoning strategies that are used vary as a function of how many unexpected findings are obtained. Furthermore, there gender differences in the way s that unexpected findings are treated: Men are more likely to assume that they know what was wrong. Women are more likely to repeat the experiment with different hypothesis and to systematically eliminate problems. Dealing with the unexpected should be a key component of our science education.

What are the (most important) unsolved problems? What should be included in an agenda for research?

Looking at how distributed reasoning works in groups. If you want a group to work the group has to have not only common goals, but also must have knowledge about the problem. Much of the work that has been done demonstrating that groups do not help in problem solving and reasoning has often given the groups arbitrary problems to solve that they do not care about. To understand how groups work in thinking and reasoning processes, research must look at groups that are involved in solving problems that the individuals in the group care about solving.

There is also a problem with our culture of the individual. A great amount of science research is done in groups and in lab settings. An interesting occurrence is that someone working in the group will come across an unexpected finding that they cannot explain. They will then bring this finding to the group and will begin the process of discovering some meaning to the finding. If you then go back several months later and ask the person about the discovery they will say that they did it all on their own, even when you have the transcripts that demonstrate that the group was actively involved in changing the way that the person thought about the problem. This notion of the “lone researcher” is definitely something that we need to move away from. One way to do this is implement group work earlier in the educational experience.

What prototypes can you point us toward where principles from the science of learning are already being applied?

I don’t think that there has been a lot of application of the science of learning principles. Analogies have been used quite a bit, but have shown mixed results in their level of effectiveness. With analogies it really depends on how they are used. There is also the MIT media-lab approach that uses computer programs to help guide learning, however unless you give people underlying structure this is not going to be very useful.

Some other examples of prototypes include case-based reasoning approaches and the work by John Anderson.

What are the major problems with or barriers to redesigning higher education? Do you have any ideas for overcoming them?

It is not simple and it is not straightforward. There are no easy answers and there is no magic wand that you can wave that will instantly make people better learners. People have unrealistic expectations about what can be done. Applying new techniques will generally only bring about modest increases in performance.

We have things such as drop out rates, especially among women in the sciences. The limited numbers of women in sciences is a great waste of talent and is an example of a problem that cannot be solved by simply changing the cognitive approach to education. There are many things that intertwine to cause this problem, issues related to the culture and to various social issues. It is important that we include motivation and social/cultural issues in our research. Merely making changes in the cognitive area of education will not necessarily change things like drop out rates. We need to look at the cultural and social issues as well. We need both situational research and educational research. The divide between these two areas is a major barrier that we need to overcome.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

Break people into small groups with a common goal. Want to avoid having the groups argue that their specific solutions are right. What would be helpful is to get people to come up a list of various options a number of different strategies that could be used in small groups and then getting these lists out to the larger groups.

(Back to top)

Frank Dempster

(Back to top)

Donald F. Dansereau

What are the findings or theories from your area of expertise that we could apply to higher education?

  1. Teaching people to use and understand non-linear representations (e.g., maps, diagrams, flow charts, animations). We now have a plethora of these non-linear presentations available to us. The linear presentation of text has been explored, but our literacy for these non-linear presentations has lagged behind. We have been able to see the effectiveness of using non-linear representations with students, and we have even gotten better outcomes with people in drug abuse counseling with map presentations, especially when their verbal skills were not high.
  2. Entertaining multiple perspectives. Nowadays, because of advances in technology, you can communicate with just about anybody. Being able to take the perspectives of others, for both communication and creativity purposes, is something that needs to be integrated into courses and curricula. We have worked with the concept of a Thought Team, a set of imaginary consultants, and have found positive effects on learning.
  3. Having people improve their ability to work collaboratively or engage in cooperative learning. I think more cooperative learning can be integrated in the adult curricula (especially in the university and in business training).

What are the (most important) unsolved problems? What should be included in an agenda for research?

There are a lot of principles and useful ideas out there, but the challenge is to develop attractive packages or tools that teachers and students can actually use and integrate into what they already do. Having some experience with school systems at the lower levels, I know teachers get this stuff all the time and unless it is packaged in a way that can be easily integrated it into what they already do, they will just ignore it. This whole notion of research on how to best get technology packaged and transferred is lagging.

What prototypes can you point us toward where principles from the science of learning are already being applied?

At the lower educational levels, cooperative learning has been something that has been pushed by getting commitments from administrators. When you get into universities, the whole transference process is different, because there is no top down authority that is going to adopt a curriculum or a practice and then make everybody use it. In universities, change has to be a grass roots kind of movement, in which you have to influence the individual faculty members. So selling it to administrators is really not going to work in the university setting. The people who have the best handle on how to get new ideas incorporated into learning are the booksellers. They are in education for the competition and they have to be very aware on how to transfer materials they develop (CD ROMs, videos, supplements to their textbook). So I think they would be good people to bring on board to find out what works.

Language learning has been a place where some of the principles have been applied pretty well, but again I’m not sure if it helps us much because many of the principles that have been applied are fairly simple—extensions of key-word mnemonics, for example.

What are the major problems with or barriers to redesigning higher education?

One barrier that needs to be overcome is the sort of “sage on the stage” mentality that most of us have. Must of us that teach, like to be up there and be the focus of attention. We need to shift to more of a “guide on the side” kind of mentality. In some sense there is a barrier that has to be respected. First of all, staged presentations is what most of us have grown up wanting to do and learning to do, so we have to be very respectful of that sort of mentality. But on the other hand we also need to try to move towards allowing more student-to-student activity and not just instructor to student activity.

The main challenge is integrating any changes we may propose with what is already being done. If we are going to try to make changes, no matter how good the principles are and no matter how effective people believe them to be, they can’t just overhaul the system. The question that people would ask is what is broke? Why make these changes? Is education as we know it not working and do you have evidence that is not working? And that is what the consumer is going to be saying. To motivate people to change, the impetus has to be something that does not challenge their very being. It has to be something that says here is some stuff that will enhance what you are already doing. We have to have “plug and play” tools, rather than programs.

Do you have any ideas for overcoming them?

How we deal with this is important. Changing education is a bottom up process. We have to somehow make contact with the individual faculty members. I was thinking about some sort of electronic mail or newsletter that could be created as a dissemination tool that gave best practices or ideas, but in addition gave opportunities for workshops and opportunities to receive videos on some of these techniques. I don’t think I have ever received a newsletter from a non-profit organization that really tried to directly say here are some new ideas, lessons, and materials that you can use in your college teaching. I get plenty of that from booksellers, but not from somebody that I would consider unbiased. Something maybe like a consumer reports that gives some really good techniques and details how they can get a hold of them. In addition, I think that the packaging of whatever we come up with is going to be critical and we are going to need advice from marketing folks and not just psychologists.

What additional questions should we be asking?

Some people are going to say what is wrong with what we are already doing. Why do I need new techniques? And that leads to how do we evaluate what is going on now? How do we decide that things need to be improved? It is okay to say I have all these principles, but a lot of instructors, especially those not in psychology or education, would say that all that psychologists do is verify common sense and that they already do all this stuff. And they may say why do I need you coming in and telling me this when I have had 20 years of experience teaching and I have learned all these concepts. So we need to have something in place that says there is a gap in what is going on already. So that’s why I try to look for instances where the technology is changing or the culture is shifting (e.g., the expansion of non-linear representations) and suggest that we need to cope with this. Then maybe we can sell teachers on those kinds of ideas because these are tools that are for new or emerging problems.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

There are probably three levels of research that should be put on agenda by granting institutions. One is to pick a couple of well-developed packages or tools or whatever we are going to call them and look at transference strategies. I don’t think we know enough about how to transfer things into colleges. So pick something that has been well established and run some regional transference studies to see if we can understand the best way to do this (i.e., newsletters, websites, or whatever it may be). The second layer is to take some less developed tools and come up with some palatable packages and evaluate them. This level would involve things that we already know to be effective and now try to make them so that they would fit into a university curriculum. The last level would be to develop new tools based on issues that seem important and principles that seem promising and test them.

(Back to top)

Rodney R. Cocking

Area of Expertise

Developmental science, specifically interested in learning processes, symbolic functioning, representational thinking, and how you can think in the abstract

What are the findings or theories from your area of expertise that we could apply to higher education?

I think that the major findings come from following five targeted areas each of which has been well researched.

  1. Structure of knowledge and memory
  2. Analyses of problem solving and reasoning
  3. From developmental psychology, we know about the early foundations of learning and how early learning relates to later learning
  4. Meta-cognitive processes
  5. A newer area that is gaining attention is looking at how culture and cultural experiences mediate learning, memory, and problem solving

These arefive areas that we know something about and that we are ready to use for influencing programs and curricula in schools.

What are the (most important) unsolved problems? What should be included in an agenda for research?

The biggest unsolved problem is in terms of the mediators of learning. For example, technology has become a major influence in our lives, both in schools and in our homes. Technology is pervading our lives, and yet we do not have a good understanding or knowledge base of how technology is a mediator of learning and understanding. I could also say that is true of the issue of culture and cultural experiences. I think that the mediators of learning are what we really need more information about: culture and technology are the two high-profile examples of this.

We need to know about conceptual change. We want research to look at how mediators influence our concept formation. We need to know how they are used to alter our preconceptions so that we can change our beliefs and change our understandings and change our behavior as we acquire more information. Cultural belief systems influence how we approach problem solving. Children come to school with a lot of notions that need to be corrected; those notions are mediated through the culture. Teachers also have their own conceptions of how children learn, as teachers gain more experience in teaching and in teaching with technology, they may need to change their conceptions of children’s learning. The big unsolved problems lie in our understanding of what it takes to change people’s preconceptions, this is for both students and for teachers. The big challenge is in the area of conceptual change and what it takes for learning to occur.

What prototypes can you point us toward where principles from the science of learning are already being applied ?

The important prototypes are the ones that have been built from what we know about meta-cognition. These prototypes come right out of basic developmental research that demonstrates the role that self-awareness plays in learning and in terms of appraising the learning processes. The prototypes that have taught children to how to ask questions and how to develop self-awareness of their learning process have been influential in many areas, including the development of mathematical programs that are based in meta-cognition. These math programs teach students how to estimate and how to diagram information and these are concepts that come out of our understanding of meta-cognition.

What are the major problems with or barriers to redesigning higher education? Do you have any ideas for overcoming them?

I think the two areas of continuity and time on task combined are the biggest problem that both teachers and school systems are faced with. If you have a curriculum that is set up with a certain fixed number of units that a teacher has to work through in a specific amount of time, then the goals and objectives are built around the number of units and the amount of time available. We know from effective learning that understanding and competency cannot be rushed. I think a definite barrier is the way we conceptualize time and continuity.

A big part of overcoming this barrier is by understanding that competency and understanding are not off-the-shelf ideas that can just be handed to the learner after a specified amount of time. You achieve that understanding, on the teachers’ part at least, by changing the instruction that teachers receive in colleges of education. We can work on this problem by instructing future teachers in areas of developmental psychology and developmental processes so that they learn about skill acquisition and competency development, and so that they develop an understanding that it takes time and time on task to develop competency.

What additional questions should we be asking?

If we are looking at student performance and what it takes to move students from being a novice in an area to being competent in an area, we need to ask what the corollaries are to moving from novice to competent performance. What are the support systems that influence this? What are aspects of the child’s family and background that correlate with this learning process? Again, we really need to look at the mediators of learning and the role that the culture and social influences play. A major correlate involves the role of transfer of learning. We need to do a better job of linking domains of knowledge and linking knowledge from context to context. The role of prior knowledge and students background knowledge and the role of culture as a mediator are all really big unanswered questions that influence students’ learning.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

I think that we really need to think long and hard about what it means to effect change in colleges of education. I mentioned that is one of the biggest barriers that we face and I think that we need to spend some time talking about the hardest problems to solve. The toughest problems are involved in changing the training teachers get. Organizational change is very hard and I would be very interested in hearing other people’s perspectives on how to go about this.

(Back to top)

Alberto Cañas

Area of Expertise

  • The use of technology in education
  • Software tools in education

What are the findings or theories from your area of expertise that we could apply to higher education?

  1. Fostering knowledge modeling to support learning both for evaluation and collaboration between students. Having some kind of a medium by which students can express what they know about a topic and share it with others.
  2. Using the Internet as means for collaboration, as opposed to a means of just posting content for the students to browse. Having students be knowledge constructors instead of just information consumers. One of the problems that I have seen is the lack of tools that are developed open environments for collaboration or knowledge construction (both at K-12 and higher education).

What are the (most important) unsolved problems? What should be included in an agenda for research?

Departments. One problem we have with the universities is that departments are very compartmentalized and many departments teach their courses and don’t really care that much about the service courses they teach. You have a course in one topic that a department teaches and it has nothing to do with the other courses from other departments. This makes it very difficult to create any kind of interdisciplinary programs. It makes it difficult to take advantage of content from different departments and put together courses.

Lectures. Another problem is that a number of students can’t seem to get away from the professor standing up and lecturing. Most professors love to stand up and lecture. They assume that the students that sit down in the class will learn from the professor lecturing and getting away from this style of teaching will be somewhat difficult.

Linearity of the Courses. Sometimes when I have a lot of students in my class, it is hard to get away from the textbook. You have the linear sequence that the textbooks gives you and you really cannot teach things in another order, mainly because you cannot really base your teaching on articles, change the order of the chapters, and because textbooks are written in a linear way. Technology can help us if we use it wisely. In most cases we are taking the technology to just put core courses on the network. The problem is that every university is coming up with its own courses. How many psychology 101 courses are we going to put on the Internet before we find out that most of them are pretty bad? We are going back to what happened many years ago when every university had to create their own textbooks because nobody was willing to use somebody else’s textbook, until we found out that most of the textbooks were useless. Then we started to share textbooks and now it is common to use somebody else’s textbook. The online content will get to the point where we will be able to pull online content from different places and create our own courses the way we want them.

Lack of Software in Education. There is very little software to support any type of collaboration or open environments for learning. Most of the tools that we use mostly come from the business area. We do not really develop many tools that foster a learning environment. We need software that allows students to create and construct a model or a simulation of something.

What prototypes can you point us toward where principles from the science of learning are already being applied (?)

We do a good job at the two extremes of education. Kindergarten is a good example because kids go and work on their own and do research and there is somebody that supervises what they are doing, but they are not into a very tight curriculum. Teachers make sure that the students get a balanced education. Students work in groups and collaborate. We do a very good job at the masters and PhD level, where there is a supervisor that makes sure that the student gets a balanced education and gets some research in some topic. Graduate seminars are not linear in the sense that you have a textbook and it is followed, rather you have discussions and you read and present articles. It is somewhere in the middle where we have large lectures that we have a big problem. Technology can help us. We can take advantage of technology to make the content of the courses available in a modular way.

Like I said I work more with K-12 schools. There is one school that I work with in Brazil that really broke down the curriculum. They do not have textbooks, math, science, or English classes, but they have tried to integrate all the courses into projects. It is probably quite an extreme and we may not get to that, but it shows some of the interesting aspects of what happens when you are trying to integrate different areas (e.g., science, math, and history together). Using this method with these kids it has been very effective, although there are some areas (e.g., algebra) that cannot be integrate it as well as others.

There is a very interesting course that is taking place down at Brazil by a colleague. She is training, through distance learning, 500 teachers (or support personnel) throughout the whole country. The distance learning is not putting all of the content of the course on the web, but rather it involves having each of the support personnel contact a teacher that has established a group and have them come up with a project. The project is the basis of the course. One of the interesting things is that they did not know what all these teachers were going to come up with as their projects. So two weeks before the course begins, they don’t really know what the real topics that are going to be covered are, but they had 50 people to support each the 500 teachers. Most of what they would do is collaboration and exchange through email and other tools. What I found interesting was that the ratio of instructor to student was 10 to 1, which is completely the opposite of what you find with distance learning courses. The ratio of most distance learning classes is much larger because you assume that a single instructor can carry on many students. I like the approach that they were doing because they have access to many people while they are teaching.

What are the major problems with or barriers to redesigning higher education? Do you have any ideas for overcoming them?

The major barrier is that the universities tend to be the most conservative organizations in terms of change. The other problem is the faculty. We are going to have to get the faculty to change. Faculty are used to the lecture modality and they will lecture as well.

What additional questions should we be asking?

I think one question we should ask is how do we evaluate whether people have learned or not. One of the problems I have with higher-level education is that a lot times classes only have one midterm and one final and that takes up the majority of the grade. This approach assumes those tests are a good way to evaluate whether people have learned or not and not everyone is a good test takers and they don’t have a different opportunity to show us what they have really learned.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

The main way to get change is to implement and show the results. You can write as much as you want, but it is not until you can show it that you can achieve change.

(Back to top)

Merry Bullock

Area of Expertise

Early cognitive development (substantive); science policy and program development (from perspective of a professional association -American Psychological Association).

What are the findings or theories from your area of expertise that we could apply to higher education?

Cognitive development, specifically the development of expert knowledge and procedures. I am interested in looking at how the acquisition of new knowledge and procedures interact with prior knowledge and how we can use prior knowledge to design curricula. Some of the theories and findings from early cognitive development that are applicable to higher education involve the development of expertise in a content area, and the development of expert procedures (“how to” strategies). What is important in this regard is to use what we know about what the person brings to the classroom in order to enhance instruction techniques and content. For example, people form very strong views early on about how the world works, including concept content and assumptions about event structures. Sometimes this understanding is incomplete, or skewed, or simply wrong – but it is not a simple matter to simply teach what is right. This entire field, labeled ‘cognitive misperceptions’ yielded some important insights about how curricula need to keep both starting and end points in mind. And now there is research that goes further and helps us understand not only how to overcome cognitive misperceptions but also how to integrate the development of expert knowledge and procedures acquired in a formal setting with everyday contexts.

What are the (most important) unsolved problems? What should be included in an agenda for research?

In thinking about the changes that are taking place in higher education two things come to mind immediately. First, there are substantial changes in the demographics of people attending higher education. Secondly, there are huge changes that have taken place in the technology that we have available. These two changes mean that the “face” of education is changing. There are tools we can use to think about these changes, and to think about how to alter curricula.

As demographics change, the structure and content of education may change as well. At an organizational level, we probably need research on the structures of higher education – to ask what sorts of models (liberal arts, community college, technical schools, apprentice models) are appropriate for what sorts of outcomes, but this is beyond the scope of psychology! At a curricular level, it would be interesting to have more comparative studies like those currently available for mathematics and reading – to be able to map out how different strategies in different settings affect learning differently. And at a very direct level, it is probably important to assure that our curricula prepare students to live in an increasingly international world – to “internationalize” the curriculum. There is cross-cultural research that can be useful here.

Technology is the current buzzword, and for good reason. Technological advances are happening incredibly rapidly – we will soon be able to have every imaginable sort of classroom available to many more people – through virtual classes, high speed cable links, and the like. We will also be able to have every imaginable sort of study materials – including all of the senses. But we don’t have the research to know how this will impact learning, much less longer term effects. We have a grand experiment going on right now, and one priority I would assign is research on learning in a technologically sophisticated environment. To think about how education might best demographic changes, we can look at the large corpus of cross-cultural research , I think that it would be helpful to look at what cross-cultural psychology can provide in terms of structuring curriculum. Cross-cultural psychologists have had to find ways to communicate concepts across different cultures and have had to find ways to transmit information to different cultures. It could be useful to look at the strategies of cross-cultural psychology as we look at reforming higher education.

An additional issue that is related to research concerns collaboration. How do we get psychology researchers to work with education researchers? Traditionally there has been very little cross-pollination between psychology research and education research. Funders and policy makers (such as NSF, NRC) have been trying for about a decade to bring findings from cognitive science to education, but the going is very slow. The two groups of researchers seem to speak different languages, to read different literatures – and although there are certainly many stunning exceptions, cutting edge cognitive research does not inform education research.

What prototypes can you point us toward where principles from the science of learning are already being applied (e.g., activities, courses, fields of study, degree programs, or entire systems)?

There are certainly people that have programs where principles from the science of learning are being applied. They are mostly in math and science education. Many of the people that are doing this kind of work are people that are attending this conference.

What are the major problems with or barriers to redesigning higher education? Do you have any ideas for overcoming them?

One of the big problems is that the application of cognitive science and the science of learning is not wide spread. Single schools and single schools systems are doing things individually, but we are not at a point where all teachers are aware of these principles. I really think that all science and math teachers should have an understanding of “lay” conceptions of the topics they teach (which give rise to cognitive misconceptions), and should have some understanding of what we know about conceptual change. In order to achieve this we need to have scientists become better at translating their research into a form that can be used by the practitioner. There needs to be a method or a format where current research findings are translated into practical teaching suggestions—a list of best teaching practices based on scientific findings. There is a lag between what is being done in research and what people use in practice. As a field, we need to get better at translating current research findings into practical approaches. We need a mechanism for getting the very best most current research into a wide spread forum, such as, getting this information into a PTA newsletter. I think that this is an area where professional organizations can play a part. I have often spent time asking scientists to write things in layman’s terms. Often though, this does not work and what you actually need is someone who can write that way, but who also understands the content area. Either way it is important to get the research into the hands of those who practice.

Institutional problems- higher education takes place in large institutions that have a certain amount of inertia. Getting large structures to change is difficult. We might benefit from getting I/O psychologists perspectives on the best methods for getting organizations to change.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

A very, very concrete list of recommendations with a specific list of tasks that need to be accomplished is needed. We need a taskmaster, and whether the tasks are given to individuals or to organizations, we need to have someone who can delegate these tasks. We need really concrete tasks with measurable outcomes. Additionally, publications that list the best practices for whatever goal it is that one wants to achieve.

(Back to top)

John Bransford

Area of Expertise

Learning sciences, technology, and teaching and the overlap among the three.

What are the findings or theories from your area of expertise that we could apply to higher education?

Many of the things that I could talk about are mentioned in the National Academy of Science’s Report, How People Learn, which came out in 1999. This report provides a really good synthesis of what we know about learning and its implications for teaching. It is focused mainly on K-12, but many of the implications could be applied to higher education. A central component to this is what I could call “The How People Learn Framework” that suggests four lenses that you want to use in any classroom situation.>

  1. Knowledge-Centered. The degree to which it is knowledge-centered, which means the degree to which the nature of the knowledge that is being taught is organized in a way that supports understanding and subsequent transfer. In many courses what is really happening is memorization masquerading as understanding.
  2. Learner-Centered. The degree to which the classroom is learner-centered. To what extent does one really know the preconceptions about the subject matter that people bring to the class as well as preconceptions that the students have of themselves as learners. To what extent is that thinking made visible and worked with and honored.
  3. Assessment-Centered. The degree to which the class is assessment-centered, and by this I do not mean that you test the students to death. But what you do, is provide frequent opportunities for the students to test their own knowledge and to see if they understand, and then you provide them with opportunities to revise. This is assessment for purposes of feedback and revision and is probably the single most important thing that we could do to facilitate learning.
  4. Community Environment. The degree to which you have created a community where people feel free to say that they do not understand and the degree to which they feel free to collaborate with one another rather than compete with one another. There are a lot of things that can contribute to this, including how you structure the grading of the class.

You could then talk about the role that technology can play at each of these levels and how technology can be used to strengthen the framework.

What are the (most important) unsolved problems? What should be included in an agenda for research?

A huge one is the problem with assessment. What we want are assessments that can really tell us about the quality of learning, yet many of the assessments that we have in place often just measure the ability to memorize information. We have some assessments that measure transfer, but the theories of transfer that underlie the assessments are really restricted. They really are what I would call sequestered problem solving because students are only given information to solve the current problem, a much stronger way is to look at whether learners have enough to information to apply what they learned in a new situation. The question becomes: how do we really assess in a way that fits new theories of transfer and is a better indicator of life-long learning? We want transfer that facilitates future learning. This is a different paradigm, looking at the assessment of transfer of knowledge rather than our current assessment of sequestered problem solving.

If you then work backwards from this perspective, you begin to ask questions relating to the way that the knowledge is structured. How can we structure the nature of knowledge and organize this information both at the level of a single course and also across the span of a four-year college curriculum that will allow for future learning? We can also look at how information is taught and how we can create the appropriate macro-structure for courses that will allow people the ability to tie the courses together. Students often do not have a good concept of how different domains of knowledge are connected: how can we help to facilitate students in developing an understanding of these connections?

What prototypes can you point us toward where principles from the science of learning are already being applied?

Chapter 9 of How People Learn has a lot of great examples of prototypes.

Some examples of people that are doing this work include:

  • Jim Minstrell and Earl Hunt—strong work on using assessment and feedback.
  • Jose Mestre’s work and his use of the class talk system to provide instant feedback.
  • My website

What are the major problems with or barriers to redesigning higher education? Do you have any ideas for overcoming them?

The biggest one is time—faculty time—this is a huge problem. The other one is faculty incentive, no matter what the rhetoric is, teaching is not considered as important as research. For most of the people who will be at this conference though, the ability to do research on one’s teaching fits our research area. If we could begin to build a community where it is valued by an academic community to combine strong domain specific-expertise with pedagogical content knowledge needed to help students learn and to assess their learning, then this would be a strong way to overcome some of the barriers. Time and isolation are some of the major barriers.

The other problem is students. You have to give students a reason for changing the structure. Some of the changes will require students to be more active and might disrupt the way that students have become accustomed to being successful, for example their mastery of multiple choice exams and such. Unless you provide students with an explanation for why you are changing the rules, they often get very upset.

What additional questions should we be asking?

How do we use technology to start to leverage all of the knowledge that is available about teaching and learning that is dispersed throughout the nation, but is not in a form that is easy for people to get and to learn from? How can we start to communicate our ideas as learning scientists in ways that make contact with professors in other disciplines who don’t know our lingo? Simply having them read our journal articles is not the best method. We have begun a project that where we have talks on the Internet. It is a whole new way of publishing that allows many different voices to be heard in a way that people are able to understand. We really need to re-think how we communicate our knowledge and how we can share our resources with one another and we have a great opportunity to do that through technology.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

We must do something other than just write a report. It would be nice to have a place for people to go that takes them on a journey and that directs them to papers and resources.

(Back to top)

Elizabeth L. Bjork

(Back to top)

Robert A. Bjork

(Back to top)

John R. Anderson

What are the findings or theories from your area of expertise that we could apply to higher education?

Our research has involved developing computer-based tutors to mathematics.   Our tutors are currently in use by about 100,000 students around the country.  They use a cognitive model of student problem solving that is capable of solving the problems in the same way students do and making the same errors.  We use these models to interpret student behavior and provide instructions appropriate to where the student is in the problem-solving situation. They are also used to implement a mastery-based system whereby students are promoted through the material as they master it. This has produced significant achievement gains.  A similar approach could be taken to instruction in higher education and indeed a little of this is already done in my courses at CMU.

What are the (most important) unsolved problems? What should be included in an agenda for research?

To be able to apply this approach it is necessary to precisely define what the goals of a course are and do a cognitive task analysis of the competences involved.  This is very time consuming and is only practical when the cost of the effort can be amortized over the benefit to many students.  Developing efficient cognitive task analysis is a major problem.

What prototypes can you point us toward where principles from the science of learning are already being applied (e.g., activities, courses, fields of study, degree programs, or entire systems)?

There are the math tutors (see http://carnegielearning.com) and at CMU we have tutors for programming an beginning parts of cognitive modeling.

What are the major problems with or barriers to redesigning higher education? Do you have any ideas for overcoming them?

I think a significant problem is coming to consensus on the goals.   There was an effort in Britain to deliver computer-based instruction in psychology.  I do not know the ultimate outcome (I could find out) but I do know that they suffered enormous problems because psychology faculty could not agree on the objectives (or how to measure them) for basic courses

What additional questions should we be asking?

How can we be precise in formulating our goals.  Your current web site speaks of "maximizing transfer to the real world", "enhancing critical thinking abilities", "encouraging the habit of life-long learning" and so does everyone else.  One would think psychologists would bring some definition to these apple-pie goals.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

There are numerous efforts afoot to reform education involving things such as new technology or new theories of instruction and curriculum.   These efforts are largely devoid of any concern with psychological theory or good empirical assessment of outcomes.   Given that these efforts have large constituencies and can command large amounts of funding,  I think it becomes important to figure out how we can get psychological concerns represented at these tables.

(Back to top)

Franca Agnoli

Area of expertise

I teach statistics and experimental design as a member of a psychology faculty in Italy. My areas of expertise are primarily in human reasoning, problem solving, and related developmental issues.

What are the findings or theories from your area of expertise that we could apply to higher education?

There is a large literature of theories and findings in human reasoning and problem solving that could be applied to higher education. Much of the research has focused on the shortcomings or failures of human reasoning. Some work, including my own, has shown that some simple training sessions can improve reasoning in these situations. For example, training with Venn diagrams helps people avoid the conjunction fallacy (this has been done with paper and pencil as well as computerized sessions).

What are the most important unsolved problems? What should be included in an agenda for research?

One of the major problems is that what is known is not transferred into the teaching curriculum, in part because university professors may not know the relevant findings. Consider, for example, university statistics courses. We know that people are better able to deal with frequencies than probabilities (very large differences in correct solution to problems dealing with conditional probabilities according to whether they were presented in a frequency format or in a probability format). I doubt that many statistics teachers (or statistics book writers) know about this result.

Even though we know a fair amount about statistical reasoning (what are the difficulties, etc), very little has trickled down to teachers of college students (or pre-college students). This is true in both the US and Europe (I can talk about Italy, but also there is recent evidence from Germany).

The same must be true for teaching many other subjects. Professors of physics, chemistry, mathematics, history, and economics are experts in their domains, but they are not experts in the factors that enable students to learn most effectively. Transferring that knowledge so that it can be used to make education more effective in all fields is a major problem.

A second problem is that even those things that are widely known and accepted are not consistently applied. For example, it is well known that people will not learn a problem solving method by watching or listening to someone else applying the method. People learn by applying the methods themselves. This is called learning by doing.

Training effects are quite large when “learning by doing” training is used in teaching statistical reasoning. For example, students have a lot of difficulty understanding probability distributions and sampling distributions. Suppose that they actually removed balls from urns or rolled dice instead of just reading about these events. They would understand these concepts much better.

The benefits of learning by doing are well established. So why do we still teach 300 students at a time in large classrooms? (At least in Europe this is the case, and at large US Universities the same). In smaller classes, students could have more hands-on experiences, either with physical objects or through use of computer simulations.

A third major problem is that we don’t teach students how to approach or solve problems like those experienced in the real world. In university classes we generally focus on controlled situations and problems that have a single correct answer. Real world problems are vastly more complex and may require making tradeoffs among many ambiguous alternatives. Solving these problems requires skills having to do with planning, scheduling and coordinating. And large real-world problems are solved by teams. But at the University rarely students learn to collaborate on a team.

If people do not learn how to collaborate in college, why do we expect them to do it effectively in the workplace? Possibly the emphasis on grades and fairness in grading among university students in the US works against collaboration in the classroom. One colleague who has taught in both Denmark and the US noted that students in Denmark preferred to work on group computer science assignments, while students in the same class in the US strongly resisted group assignments.

That brings me to note the increasing diversity of the workforce and the need to prepare students to work in that kind of environment. People from different cultural backgrounds will have to solve problems working together in a team (one in 10 US residents is foreign born). We know that women experience difficulties when entering professions viewed as culturally appropriate to men; maybe university educational programs can change this. The acceptance of different perspectives in the workforce should start at the University.

What are the major problems with or barriers to redesigning higher education? Do you have any ideas for overcoming them?

In Europe some major revisions are in progress of the higher education system at both the undergraduate and graduate levels. The primary motivation for these revisions is to create a more uniform system across Europe that will make it easier for graduates of a university in one country to work or continue their studies in another country. With so much emphasis on this major restructuring, there may be little near term interest in improving educational practices.

(Back to top)

Phillip L. Ackerman

What are the findings or theories from your area of expertise that we could apply to higher education?

The investment prospective of adult intellectual development that builds on work of Raymond Cattell – I call it “PPIK” because it combines intelligence-as-Process, Personality, Interests, and intelligence-as-Knowledge (Ackerman, 1996). The PPIK approach builds on the interactions among traits that people have as a basis for understanding individual differences in knowledge as a function of the direction and level of cognitive resources that people devote to acquiring academic knowledge and information about the world around. This approach differs from “content-free” approaches to adult intellect and knowledge acquisition (e.g., intelligence as “g” a general ability). That is, in this framework, intellectual abilities are contextualized in particular domains – which differ between individuals and within individuals over time.

What are the (most important) unsolved problems?

We have identified “positive achieving trait complexes” and a “negative achieving trait complexes;” the right configuration of ability-personality-interests leads to higher levels of achievement. People are more likely to fail at post-secondary education (or are less likely to pursue educational opportunities in adulthood) when they are characterized by the negative achieving trait complexes. An unsolved problem is the ability to identify these different achievement trait complexes in people in order to create an intervention that would lead to success in the education system.

There are substantial gender differences in knowledge and achievement. An example can be found in the College Board Advanced Placement (AP) Exams. In the last year, 574,000 men and 667,000 women took AP tests, however only 217,000 females compared to 225,000 men received passing scores of a 4 or 5. That is, 92,000 more females took the AP exams, but they got 8,000 fewer passing scores than males did. These are fundamentally important differences. It is difficult to understand the reasons for sex differences in achievement because the female and male students are coming from the same classes, have the same education experience, and, we assume, the same amount of motivation. This issue does not get addressed very often. These differences in AP scores are across most subject domains (except, for example, in the foreign language domains). Such results have a knock-on effect when it comes to performance in advanced levels of collegiate specialization, as can be seen by gender differences in the GRE “subject” tests, and in the subsequent “under-representation” of women at Ph.D. levels of education in the United States.

A third unsolved problem is in the area of motivational skills, which are addressed in the work done by Ruth Kanfer (e.g., Kanfer & Heggestad, 1997). There are two types of motivational skills that promote different types of educational outcomes. The first is “emotion-control skills;” these skills come into play when individuals are confronted with different novel tasks. These skills allow learners to keep frustration and anxiety at bay while they work through the problem. This is often a major problem for middle-aged learners. People with high emotion-control skills are able to better focus their attention on learning when struggling through a new problem or skill to learn. The second type is called “motivation control skills;” these skills allow students to push on past the point of basic understanding of a problem. They allow for the continued motivation to keep working through problems to get an even deeper level of understanding. As psychologists, we do not pay adequate attention to these motivational skills in the learning environment. These are motivational skills that can be trained, but often are not because they tend to be content-domain independent.

What should be included in an agenda for research?

First, we need to find better ways to answer questions about positive and negative achieving trait complexes; how can we identify people at risk, and give them the extra skills that they need to succeed, and how do we identify those individuals with high achievement trait complexes.

Second, we need to have a better understanding of why men and women achieve different levels of domain knowledge. There is substantial folklore as to why this is true (and a highly controversial body of empirical research on how schools treat girls and women), but there is not a lot of research that can help explain these differences in knowledge acquisition. It is not a question of whether women can acquire the knowledge, but instead why they do not acquire the same level of knowledge as men, especially within constrained environments.

Finally we need to have specific training interventions that include issues concerning emotion control and motivation control that can be developed to help students. This is especially important for at-risk students.

What prototypes can you point us toward where principles from the science of learning are already being applied (e.g., activities, courses, fields of study, degree programs, or entire systems)?

Within higher educational there is not a lot of intervention that takes into account individual differences in combinations of abilities-personality-interests. Earlier work by Lee J. Cronbach, Richard E. Snow, and their colleagues suggested that aptitude-treatment interactions were likely to be a function of trait complexes (combinations of traits that facilitate or impede learning). However, because of the complexity of this kind of work, few researchers in recent years have attempted to address these important issues. With the increased availability of intelligent computer-aided instruction, it is indeed possible to implement theory-based interventions in an individualized fashion.

What are the major problems with or barriers to redesigning higher education?

There is too large a focus on trendy things, like critical thinking. At the elementary and secondary school level, educators are attending too much to issues such as self-esteem or, for example, interpersonal and intrapersonal intelligence. In contrast, they spend far too little time on developing knowledge and expertise. Skills such as critical thinking require a basic foundation of content knowledge, which is too often lacking at this level. In post-secondary educational settings, students are often confronted with the need to acquire extensive content knowledge without the necessary skills for acquisition and retention. Moreover, educational institutions make too little use of potentially available information about the students. SAT scores, for example, are generally ignored subsequent to initial matriculation. Personality and interest assessments typically never enter into advising and curricular decision-making.

Another problem that is particularly applicable to non-traditional students is our inability to build on students’ existing knowledge. Non-traditional students come to the university with a large base of knowledge, but we do not build on it; at traditional 4-year institutions, we treat them the same as any other student. There is a real need to identify students’ base knowledge profiles in order to further build on them. We need to structure instruction that builds on the domain knowledge that students have.

Do you have any ideas for overcoming them?

If we focus more on what our students know, we can substantially improve their learning.

We spend too much time studying “maximal” performance situations, such as the SAT, when what we are really interested in is long term knowledge retention and use, which is better represented in “typical” performance situations. Predicting whether a student succeeds beyond the first year of college/university study, remains very much uncharted territory.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

There is a real disconnect between “cognitive/experimental” psychology and education. In many ways, cognitive/experimental psychology has failed to concern itself with real-world learning and performance issues. Bringing educators, policy makers, and applied-oriented cognitive psychologists together may yield an important increment to both basic and applied research concerns.

Gender differences in educational settings – especially at the level of higher education are currently understudied. It is an easily identified problem that needs more research, especially in light of the well-researched basis for gender differences in basic cognitive and affective traits.

Unfortunately, though, many of the findings from empirical research suggest that there are no “easy fixes” to problems of education and achievement. There is no single button to push or medium in which to present information that represents the perfect educational environment for everyone. Methods must be found that orient students toward knowledge acquisition, and incrementally improve educational outcomes. Such solutions are not flashy – they cannot readily be encapsulated in slogans on phrases. Empirical research aimed at increasing student achievement and retention in higher education must be carefully scaffold in the same way that principled knowledge is.

(Back to top)