The Statistical Interpretation of Entropy
Part II: The Second Law of Thermodynamics

Larger Numbers of Coins

Recall the coin flip experiment from Part I. Flipping those coins gets pretty tedious after a while. It would be even worse if we used a larger number of coins. So to avoid this problem we will instead use a computer simulation of our coin model. Run the ejs_StatisticalInterpretationOfEntropy.jar program (if it’s not already running). In Part II, double-click the green arrow next to CoinFlipEquilibrium to run the simulation. You should see two windows. CoinFlip provides a graphical depiction of our model system, with red heads and blue tails. Number of Heads/Tails shows a plot of the number of heads/tails as a function of how many coin flips have been performed.

  1. Set the number of coins to 20 (make sure to hit enter after typing the new number). Click the Start button to run the simulation. Let the simulation run until it has done at least 200 flips, then click Pause. Look at the Number of Heads/Tails graph. Do the results resemble the results of your experiment? Explain.
  2. Click the Show Entropy Plot box to display a plot of the entropy of the system vs. the number of coin flips. Look carefully at the Entropy plot. Describe the overall behavior of the model system’s entropy. Does the entropy ever decrease, even just briefly?
  3. Change the number of coins to 200. Run the simulation until it has done at least 400 flips. In what way are these results similar to the results for 20 coins? In what way are they different?
  4. Change the number of coins to 2000. To make this simulations run faster, set the Flips per Step to 10. Run the simulation until it has done at least 4000 flips. Which of the following statements best describes the Number of Heads/Tails graph?
    1. The number of heads fluctuates wildly between 0 and 2000 the whole time.
    2. The number of heads decreases steadily to about 1000, but continues to show significant fluctuations about this value.
    3. The number of heads decreases steadily to about 1000 and basically stays there. Any fluctuations are hardly noticeable.
    4. The number of heads decreases to about 1000 and then increases back up to 2000, then decreases again, etc.
  5. Which of the following statements best describes the Entropy graph?
    1. The entropy fluctuates wildly the whole time.
    2. The entropy increases steadily to a maximum value, but then occasionally dips noticeably below that value.
    3. The entropy increases steadily to a maximum value and then stays constant at that value with no noticeable dips.
    4. The entropy increases to a maximum value, decreases back to zero, increases, etc.
  6. Based on these simulations, would you say that our model follows the Law of Entropy better when there are few coins or when there are many coins in the system?
  7. Use your answer to question 19 from the ”Row of Coins” section of Part I to explain why the fluctuations become less noticeable as we increase the number of coins in the system.
  8. What behavior would you expect to see if there were 1024 coins (just as there might be 1024 molecules in a gas)?
    1. The system would approach the equilibrium macrostate, but would exhibit large fluctuations away from equilibrium.
    2. The system would approach the equilibrium macrostate while exhibiting small but noticeable fluctuations away from equilibrium.
    3. The system would approach the equilibrium macrostate smoothly with no noticeable fluctuations away from equilibrium.
    4. The system would bounce around randomly among all of the possible macrostates.

An Ideal Gas (in a Box)

Close out the CoinFlipEquilibrium simulation by closing the CoinFlip window. We’ve done enough with our toy model involving coins. Now let’s look at something a little closer to the real world. We will consider an ideal gas in a box. An ideal gas consists of molecules that don’t interact with each other in any way, but simply fly around exhibiting inertial motion until they hit a wall and bounce off. We will assign each molecule a random initial velocity (with completely random direction and random speed assigned according to something called the Maxwell-Boltzmann distribution), but we will start with all of the molecules on the left side of the box at randomly assigned positions. (Note: you may be wondering how the molecules got to these random positions with these random velocities. One way to achieve this would be to allow the molecules to interact with each other, which would make this system chaotic and effectively randomize the positions and velocities.) What will happen when we remove the barrier in the center of the box and let the molecules move freely from one side to the other?

In the menu on the left double-click the green arrow next to IdealGasExpansion. This should pop up two windows. The Ideal Gas in a Box window shows an animation of the particles bouncing around in the box (note that the left side of the box is red and the right side is blue). The Particle Numbers window shows a plot of the number of particles on the left (in red) and on the right (in blue) as a function of time.

  1. Set the number of particles to 20. Run the simulation until the time reaches about 30, then pause it. Which of the following statements best describes the results of the simulation?
    1. Most of the particles stayed on the left side of the box and never went to the right side.
    2. Most of the particles moved to the right side of the box and then came back to the left side as a group.
    3. The particles spread out until there were about equal numbers in each side, but the numbers showed significant fluctuations about this equilibrium state.
    4. The particles spread out until there were the same number of each side and then maintained this equilibrium state without noticeable fluctuations.
  2. Now we can try to relate the behavior of this ideal gas to the behavior of our coin model. We can think of each particle of the gas being represented by a coin, with heads indicating the particle is on the left side of the box and tails indicating it is on the right side. The passage a particle from one side of the box to the other is then represented by flipping over the corresponding coin. How do the results of this simulation compare with the results of your 20 coin experiment in Part I (and the 20 coin simulation above)? How are they similar? How are they different?
  3. Now set the number of particles to 200. Run the simulation until the time reaches about 50, then pause. How are these results different from the 20 particle simulation? Comment on the size of the fluctuations relative to the size of the system. You should try the simulation with 2000 particles to make sure you can see the relevant trend.
  4. Consider all three ideal gas simulations. Based on what you learned from the coin model we studied earlier do you think the entropy of the system generally increased, decreased, or stayed constant during the initial phase of the simulation? Explain your answer by making an analogy to the results of the coin experiments.
  5. It is important to recognize that there are some differences between our simple coin model and an ideal gas. In the coin model we can specify the microstate by stating whether each coin is heads or tails. But to specify the microstate of our ideal gas we must give the position and velocity of each particle. In the exercises above we have focused on whether each particle is on the right or left side of the box, but in doing so we have ignored a great deal of information about the particles in the gas. So our ideal gas doesn’t quite behave the same way as our row of coins, and therefore the analogy you used to decide about the entropy of the gas is not exact. In fact, the entropy of an ideal gas depends on the temperature of the gas and the volume it occupies. At constant temperature (as in our simulations) the entropy of the gas increases with volume. Does this change your answer to the previous question? Explain why or why not.
  6. Remember that entropy is really just a measure of the number of microstates in a given macrostate (S = lnΩ). Which of the following macrostates of our gas in a box do you think has the most microstates?
    1. The particles are spread evenly through the left side of the box (with no molecules on the right side).
    2. The particles are spread evenly through the left two-thirds of the box.
    3. The particles are spread evenly through the left three-fourths of the box.
    4. The particles are spread evenly through the entire box.
  7. Which of the above macrostates has the fewest microstates?

Entropy and Heating

Now let’s try to see how the increase of entropy connects to another version of the Second Law of Thermodynamics, namely:

Energy will only flow spontaneously from an object with a higher temperature to an object with a lower temperature. It will not spontaneously flow in the opposite direction.

We will stick with the ideal gas in a box, but this time we are going to have two gases. We will examine a system that starts off with a cold gas in the left side of the box and a hot gas (composed of the same type of molecules, for simplicity) in the right side, with a barrier between the two. Recall that temperature is really a measure of the average kinetic energy of the molecules in the gas, so the molecules on the right will be moving slower on average than the molecules in the left side. What happens when we remove the barrier?

Close the IdealGasExpansion simulation (if it’s still open). Double-click the green arrow next to HotAndColdIdealGases. The Hot/Cold Gas window shows an animation of the gas, which starts off with the cold (black) particles on the left and the hot (green) particles on the right. Set the number of green and black particles to 200. Click the Temp Plots box to show a plot of the temperature of the gas in each side of the box (the temperature of the left side is in red, that of the right side in blue). The temperature of each side is calculated by first finding the average kinetic energy of all the molecules in that side of the box.

  1. Click Start to run the animation and let it run until the time reaches about 50. (Note: you can speed up the simulation by moving the Speed slider to the right.) What happens to the temperatures of the two sides?
  2. Once the two sides of the box reach the same temperature, do they both stay at that temperature or does the temperature fluctuate?
  3. When the two sides of the box reach the same temperature, how does that temperature compare to the starting temperatures of the two sides? Be as precise as possible in your answer.
  4. Recall your conclusions about the entropy of the ideal gas in the last section. Did the entropy of the hot gas increase during this simulation? Did the entropy of the cold gas increase? The entropy of the whole system is just the sum of the entropies of the two gases. So did the entropy of the system increase? In other words, did the behavior of this system conform to the entropy version of the Second Law?
  5. Which of the following macrostates of this system has the most microstates?
    1. All the hot particles are on one side and all the cold particles are on the other.
    2. All of the particles (hot and cold) are on one side.
    3. The hot and cold particles are mixed together with about half of each type on each side of the box.
  6. Consider the flow of thermal energy in this system. Did the behavior of this system conform to the heating version of the Second Law? Explain.
  7. Real gases don’t behave like ideal gases except under certain restrictive conditions (very low density). Instead, the molecules of real gases interact with each other and exchange energy. If a fast-moving molecule collides with a slow-moving molecules is it more likely that the fast-moving molecule will give some energy to the slow-moving molecule, or that the slow-moving molecule will give some energy to the fast-moving molecule? After MANY such interactions, what will happen to the speed of the molecules in the gas?
  8. Whether the energy is transfered by the motion of molecules (as in the ideal gas) or by interactions (as in a real gas), in which situation is the entropy of the gas greatest?
    1. When the energy is spread evenly among all the molecules and all the molecules are clustered in a small region of the box.
    2. When the energy is concentrated in a small number of molecules that are spread evenly throughout the box.
    3. When the energy is spread evenly among all the molecules and all the molecules are spread evenly throughout the box.
    4. When the energy is concentrated in a small number of molecules which are clustered in a small region of the box.

Historical Objections to Boltzmann’s Ideas

Boltzmann originally used the motion of molecules to derive the second law in 1872.1 This derivation made it appear that the second law is absolute, that the entropy of an isolated system can never decrease. But as we have seen Boltzmann’s later ideas about entropy (first presented in 1877) indicate that the second law is only very likely to hold true. In fact, we have seen violations of the second law in the models we have studied. What made Boltzmann abandon the view that the second law is absolute?

  1. As early as 1869 James Clerk Maxwell had expressed some doubts about the absolute nature of the second law. He imagined a gas contained within a box and monitored by an entity (later known as Maxwell’s Demon) that would allow only fast moving molecules to move from the right side of the box to the left side, and only slow moving molecules to move the other way. To see the effects of Maxwell’s Demon double-click the green arrow next to IdealGasMaxwellsDemon. Set the number of particles to 200 and show the Temp Plots. Run the simulation for a while, then click Demon On and let the simulation run for a while longer. Describe what happens to the temperature of the two sides of the box after you turn the Demon on.
  2. Does the behavior of the gas appear to violate the Second Law? Explain how you can tell.
  3. Consider this: does the gas in this simulation really constitute an isolated system? In other words, does it interact with anything other than the box? If so, what?
  4. For a full accounting of the change in entropy for this system we would need to keep track of the entropy of the Demon itself, which we don’t know how to do. But Maxwell’s concern was that the natural motion of the gas molecules might mimic the effects of the Demon. While it is hard to argue that this is impossible, do you think that the kind of motion shown in this simulation is likely to occur (without the intervention of Maxwell’s Demon)?
  5. Now turn off the Demon (uncheck the Demon On box) and let the simulation continue to run. Describe what happens to the temperature of the two sides of the box after the Demon is turned off. After you finish this you can quit the Maxwell’s Demon simulation.
  6. Another objection to Boltzmann’s H-theorem was presented by Josef Loschmidt in 1876. He pointed out that the law of Newtonian mechanics are time reversible, which means that they work equally well backwards in time as they do forwards. If a certain set of motions leads to an increase in entropy, then the time-reversed motion will lead to a decrease in entropy. Both sets of motions are allowed by Newton’s Laws. One way to produce these time-reversed motions is to reverse the direction of motion of each molecule. The motion of the system after the reversal will be a time-reversed version of the motion before the reversal. To visualize this effect run the IdealGasExpansion simulation again. Let the simulation run until the gas reaches equilibrium, then hit the Reverse button. Let the simulation run for longer than you let it run before hitting Reverse. Describe what happens to the gas.
  7. After the velocities are reversed, what happens to the entropy of the gas? Does this constitute a genuine violation of the Second Law?
  8. Close the IdealGasExpansion simulation and run the HotAndColdIdealGases simulation again. Use 200 particles of each type and show the Temp Plots. Run the simulation and let it go until time reaches about 20, then hit Reverse. Watch the simulation for a while. What happens to the temperature of the two sides of the box after the velocities are reversed? Does this constitute a violation of the Second Law?
  9. These objections led Boltzmann to reformulate his interpretation of the Second Law. In 1877 he presented the statistical approach to entropy that we have been examining. He admitted that violations of the second law are possible but argued that such violations would be highly unlikely and would be short-lived in any macroscopic system. Based on what you have seen in the coin flip model and the computer simulations, would you agree that violations of the Second Law are unlikely to occur in a gas with 1024 molecules? Are such violations possible?
  10. Consider what a radical shift this is in the concept of a “law of nature.” Are we really justified in calling the Second Law of Thermodynamics a “Law” if it is just a statement of probability? Defend your answer.

The Scope of the Second Law

Like all physical theories the Second Law of Thermodynamics has limited scope. Let’s explore that scope a little bit.

  1. In our coin experiment we found that with 20 coins (and to a lesser extent even with 200 coins) we got occasional large fluctuations in the number of heads that resulted in a decrease in the entropy of the system. Boltzmann argued that noticeable fluctuations would essentially never occur in a gas of 1024 molecules. What does this example tell us about the scope of the Second Law?
  2. We have seen that a gas interacting with Maxwell’s Demon can appear to violate the Second Law if we ignore the entropy of the demon itself. Use this example to discuss why we must restrict the Second Law to isolated systems. Why is it that the entropy of a system can decrease if that system is not isolated? If we expand the system to include everything that is interacting with the original system (possibly to include the entire Universe), could the entropy of this expanded system decrease?
  3. Evolution of life on Earth shows a clear tendency to produce more organized (ordered) structures over time. Does this violate the Second Law? If the entropy of all living things on Earth is decreasing over time, what else must be happening according to the Second Law?

1Actually what he derives was a theorem that has come to be known as his “H-theorem,” but he interpreted this theorem in a way that made it equivalent to the second law.