What prompted you and your colleagues to conduct this research on critical thinking in China and the U.S.?

I’m a development economist interested in the long-run growth of China. When countries are poor and try to become middle income, like China did from the 1980s and 1990s, there are lots of sources of growth: you can build roads, you can change incentives, you can invest, and in turn you can grow very fast. China grew 10 percent every year for 30 years in a row. After a country becomes middle income and tries to move to high income, and after it is high income and tries to maintain growth, the source of long-run growth is different. For the past 50 years, the U.S. has grown at 2-3% per year. We’ve grown on the basis of innovation, which is based on human capital. Countries that are middle income or high income need to have labor forces that are endowed with human capital if they want to grow. 

There are two parts to that. Most of my work is on the human capital of the entire labor force. China has huge inequalities, with especially high inequalities in its education system and its healthcare system. Inequality is also great between rural and urban populations. It is very dangerous when massive amounts of the labor force are uneducated and do not have the skills to thrive and contribute to a high-end economy. It will drag the economy down. Most of my work has been on this problem. 

On the upper-end of the scale, China may have enough talent coming out of college. The college graduates from the upper-end of the education spectrum will be able to enter into society and generate innovation and growth. However, China doesn’t have the low-end and has massive amounts of poorly educated labor forces as an anchor. Even if you have this driving force on the upper-end, namely this highly educated professional class that’s innovative and can solve critical problems, the anchor is going to hold you back. You need both. 

The question of critical thinking is the second part. Does China have this engine from the upper-end of its education system? Is it producing graduates that will be able to be innovative and create productivity gains in the economy, and solve problems in business and in society. We are interested in whether China’s upper-high school and college system are producing graduates that can do that. 

What were the most surprising findings of the study? Are the findings newsworthy?

News should be a little bit surprising; otherwise it’s not news. 

These students are all computer science and electrical engineering majors. This is a national representative sample of China’s research universities; we exclude community colleges and vocational colleges. It is a national university sample, so it has elite and non-elite universities in it. We tested freshman and juniors. We tested kids not in only math and physics. We tested the amount of education, the amount of learning, and the degree of critical thinking they’ve had through high school. 

The findings and the study offer good news and bad news. The good news is that China’s high school is remarkable, not only in the amount of math and physics that it can deliver, but Chinese students also have very high levels of critical thinking as they enter college. It’s hard to say, but our calculations suggest it is probably two to three years ahead of U.S. students. We might think that the elite – the Andover’s and Exeter’s of China might be able to produce that – but we were surprised that it was systematic across the entire spectrum. China is really doing something in high school – and I don’t want to say its right or wrong – they are doing something in high school that teaches students math, physics and critical thinking. So that is the first surprising study. 

The second one is the bad news that Chinese colleges don’t teach anything. And they actually have lower scores in math and physics, and there is no improvement in critical thinking from the time they are freshman until they are juniors. They get this spectacular high school system, but then go to sleep when they get to college. Those are the two surprising findings in the study.

Why does the study focus only on computer science and engineering? Do you think the results would be different if more than just computer science/engineering were measured?

Yes, it’s a good point. This is the difficulty of trying to do the standardized test in college. Everyone has a different major, so what should you test them in? The PISA test is a standardized test that is given around the world every two or three years to secondary school students. 

(Parenthetically: It is the one that Shanghai blew everyone out of the water in 2012, and that’s a little of what motivated this paper too. Of course, it was Shanghai reporting their data, while in the United States we have a national representative sample.  In other words, here we were testing rural Mississippi and Southside Chicago, as well as Palo Alto, Santa Monica, and Andover.)

In high school everyone is taking general classes – math, reading, English and history. When you get into college people split up into majors, so what should we test them on if we want to compare the United States and China? You have to choose a specific major and even within majors you have to be careful: is it quantitative computer science or is it empirical or even theoretical computer science? We were very careful when matching majors across universities within China and then between the United States and China. So, why computer science and electrical engineering? We are interested in whether China is going to develop this professional class that can solve problems and innovate. These are two of the areas where a lot of innovations in the world have taken place over the last 20 to 30 years. So we thought what better place to look for that than there.

What happens if we had studied instead economics or psychology? Would the results have been the same? I don’t know the answer to that because we didn’t do that. But we were a little worried that we were just going to get the elite students in China and the elite students in the U.S. What we found in China was that the college entrance exam scores were actually just a little bit above average; they are not the elite students of China. In fact, from a pure college entrance exam point of view these are pretty average, typical students. How should we interpret that? Would we find the same results, if we had studied students of economics, history, physics or math? I don’t know. We would have to do more research in the future on different subjects to know.

How does the national college entrance exam, Gaokao, influence the focus of secondary education?  Specifically, the conventional wisdom blames Gaokao as a system that encourages “teaching to the test,” which destroys critical thinking and creative learning.  But your study appears to challenge this view. Do you believe that Gaokao is not the real culprit? 

The first thing to be careful about is to not put critical thinking and creativity as substitutes for one another. I make that mistake sometimes and my colleague, Prashant Loyalka, who is the principle investigator in this study, is an expert in this area and he always corrects me. In the next round we’re going to be examining creativity and not just critical thinking. So yes, we were a little surprised that China did so well on critical thinking. Critical thinking has a lot of logic components and math components. I like to say it’s like the old GRE test where there was a math, English and analytical section. Critical thinking is like the analytical section. There is a lot of logic and systematic thinking that apparently the Chinese system is good at teaching, even though their objective is to get kids ready for the Gaokao. Of course, you can’t memorize 450 textbooks; they do have to teach stuff and students have to accumulate knowledge and skills to be able to thrive in this test. So it is cramming, it is teaching to the test, but it is teaching to a mammoth test. China is obviously doing something and if you had to take away a surprising fact, this is probably it, so I like the way you asked that question. 

I don’t think we have explored all the costs of this system. I don’t want to be Gaokao apologist and say that we should change the U.S. system to be like that. We don’t know the impact on creativity and we don’t know the impact on burnout. Why do we see learning stop at 12th grade, which basically it does? What are the benefits of learning a little slower when you are in high school and continuing to learn through college, which is what we see the U.S. system doing? To be clear, I’m not saying the U.S. system is better. We just don’t know.

What do you attribute to the decline of critical thinking in Chinese colleges and universities? Why do American colleges and universities appear to do a better job in cultivating critical thinking?

It is a whole range of incentive problems between the supply and demand sides. Between the demand for learning from students; the demand that their teachers teach them something; the demand that they are in institutions that are continuing to teach them; and then the demand on the system for the suppliers of education for the teachers, what universities expect out of professors, and how they evaluate them.  All of these things line up to everyone graduating because college is so restrictive. In the U.S., 80 percent of kids start college. We have a very low graduation rate and then we have a lot of kids that go back to college for community college later. The U.S. is a hard case for comparison. Europe has probably 60 percent graduation rate from college. 

In China only 20 percent of the cohort is going to college. China artificially restricts the number of students and everybody will get a good job when they get out. So why work? The kids know that college is just a signal. If you go to an elite college, you’re an elite student and companies are going to want you. Nobody looks at the students’ grades. The universities basically let every single person graduate. Graduation rates from colleges are 99 or 98 percent in China. There is zero incentive for anybody to do anything, and they don’t.

In the U.S., I think that we have a different ethos in our university system. We don’t have the greatest preschool, elementary school, junior high, or high school systems. We are so decentralized in public education. There are 40,000 school districts all offering different curriculums and having different amounts of resources at hand. 

Our colleges are very different, however. Students are paying a lot of money to go to college and the returns to a college education are high, but companies and graduate schools look at college grades and accomplishments. Colleges, in turn, put big demands on teachers to teach well, not in every college, but in many. There is a huge competition among private universities and private colleges, which spills over to competition between private and public colleges, between public and public colleges. And that competition forces universities to make sure they’re delivering high quality education. I know you worked harder, infinitely harder, to get into college than I did. And you worked harder than my son who is 33 years old. It’s becoming more and more competitive. But it’s not China! It’s not the Gaokao system. I’m sure you were tired at the end of your senior year in high school, but you weren’t totally burnt out. 

How do you think your study can help Chinese colleges and universities improve the cultivation of critical thinking? Specifically, how do you think the Chinese education system would have to change in order for students to continue improving their critical thinking skills after entering college?

This is the ten thousand dollar question. The findings of the study do not show that no one is learning critical thinking; it shows that on average critical thinking is not being learned in college. There are schools, professors and students where critical thinking goes up, there are schools, professors and students where critical thinking goes down, and there are ones with no change. The answer is implicit in what I said about China’s problems. Number one, the biggest component of spending on education in China is on higher education and we hope that in publishing this type of work somebody sits up and says, “we have a problem here!” The fact is that when we give these results everybody in China laughs and says, “we knew that!” However, until this study nobody has ever quantified it. The findings confirmed the suspicions. 

The second thing is that when professors are doing too much research, their students don’t learn anything. The more papers they publish, the worse their kids do on exams. In the United States, it’s the exact opposite. The empirical facts are that professors who are more engaged in research in the U.S. tend to be better teachers and their students learn more. It’s probably due to the nature of research that we do and the roles of students in the research. However, in China students tend to either be ignored by the busy professors or they are given work on little components of the research. The professor is so busy that he or she skips class or shirks on teaching, and this turns into an absence of learning. We have work that shows this. In general it is this lack of commitment to competition and the evaluation of students. It’s a whole makeover of the ethos of what colleges must demand of students and of professors. That’s the most basic thing, but there are also some very specific things that can happen too.

Would you say that things need to change more in college? Or is it the focus of high school that needs to change more?

Probably both. I don’t think we know if it’s a burnout issue, an institutional problem at the college level, or, most likely, both. If that is the case, it would mean we would want to change both of them and decrease the role of the Gaokao somewhat. South Korea, Japan, Taiwan, Hong Kong, and Singapore are the other East Asian countries that are very competitively exam-focused. They have the same problem. 

We are trying to run the same exam in Japan and South Korea in the next year or two. Our colleagues in those countries say, “you’re going find the exact same things here.” So this is not just a China problem. They say it’s all about this competitive entrance exam, but it must also be about the university system once you get in. 

It is going to be really hard to change the Gaokao admissions system. People really believe in it. There is an absence of trust on so many fronts in China. What we see coming back over and over again is people saying, “the one thing you can trust is that the Gaokao is fair.” I think it is fair given the amount of education students get up until the day they take it. People don’t cheat on the Gaokao so it’s going to be really hard to rely on letters of recommendation, community service, and AP classes. Reform will be slow. More of an effort can be made at the university level first just because of this problem. If pure and simple burnout is the root of the problem, then a change in the college system is not going to help very much.  

Finally, your study shows that American high-school students actually lag behind their Chinese counterparts in critical thinking.  What are the possible causes?  How can American students improve their critical thinking before college?

I don’t know if it’s all bad that we are three years behind. There is bad news and good news in that. I don’t think we are measuring enough. Do we measure interest in learning, creativity, understanding or commitment to society, or community? I know U.S. students have a better understanding of society and communities, even if this occurs through forced community service just to check boxes. For a lot of kids that take service seriously, it is an eye opening experience. 

The United States’s problem is the inequality of education. Our good high school systems do a tremendous job of preparing our kids in a very broad way for college and for life. But that’s not high school in West Oakland or Harlem or rural Mississippi. Everything is locally funded or most things are funded locally in the U.S., so poor people live where property values are low and poverty has social problems that come with it. A black freshman going into an inner-city high school has a higher chance of being in jail than he does graduating from high school. Is that preparing them for creativity? Of course, we don’t even pick those students up in this exam because we only look at those that have gotten into engineering programs. We don’t pick up all the Chinese who are low cognition either and they certainly are not being prepared, just as our students from poor inner city schools certainly do not have the skills that are going to allow them to thrive. That’s a big part of the problem—the first part of the problem. I guarantee that these issues are going to be front and center. It is more than just how much critical thinking we are teaching Palo Alto kids. That’s still going to be important, probably even more important in the future, but this polarization will rise exponentially as an important societal factor.