The Statistical Interpretation of Entropy
Part II: The Second Law of Thermodynamics
Larger Numbers of Coins
Recall the coin flip experiment from Part I. Flipping those coins gets pretty tedious after a while. It would be even
worse if we used a larger number of coins. So to avoid this problem we will instead use a computer simulation of our
coin model. Run the ejs_StatisticalInterpretationOfEntropy.jar program (if it’s not already running). In
Part II, double-click the green arrow next to CoinFlipEquilibrium to run the simulation. You should see two
windows. CoinFlip provides a graphical depiction of our model system, with red heads and blue tails. Number of
Heads/Tails shows a plot of the number of heads/tails as a function of how many coin flips have been
performed.
- Set the number of coins to 20 (make sure to hit enter after typing the new number). Click the
Start button to run the simulation. Let the simulation run until it has done at least 200 flips, then
click Pause. Look at the Number of Heads/Tails graph. Do the results resemble the results of your
experiment? Explain.
- Click the Show Entropy Plot box to display a plot of the entropy of the system vs. the number of coin
flips. Look carefully at the Entropy plot. Describe the overall behavior of the model system’s entropy.
Does the entropy ever decrease, even just briefly?
- Change the number of coins to 200. Run the simulation until it has done at least 400 flips. In what
way are these results similar to the results for 20 coins? In what way are they different?
- Change the number of coins to 2000. To make this simulations run faster, set the Flips per Step to 10. Run the
simulation until it has done at least 4000 flips. Which of the following statements best describes the Number of
Heads/Tails graph?
- The number of heads fluctuates wildly between 0 and 2000 the whole time.
- The number of heads decreases steadily to about 1000, but continues to show significant
fluctuations about this value.
- The number of heads decreases steadily to about 1000 and basically stays there. Any fluctuations
are hardly noticeable.
- The number of heads decreases to about 1000 and then increases back up to 2000, then decreases
again, etc.
- Which of the following statements best describes the Entropy graph?
- The entropy fluctuates wildly the whole time.
- The entropy increases steadily to a maximum value, but then occasionally dips noticeably below
that value.
- The entropy increases steadily to a maximum value and then stays constant at that value with
no noticeable dips.
- The entropy increases to a maximum value, decreases back to zero, increases, etc.
- Based on these simulations, would you say that our model follows the Law of Entropy better when there are
few coins or when there are many coins in the system?
- Use your answer to question 19 from the ”Row of Coins” section of Part I to explain why the fluctuations
become less noticeable as we increase the number of coins in the system.
- What behavior would you expect to see if there were 1024 coins (just as there might be 1024 molecules in a
gas)?
- The system would approach the equilibrium macrostate, but would exhibit large fluctuations away
from equilibrium.
- The system would approach the equilibrium macrostate while exhibiting small but noticeable
fluctuations away from equilibrium.
- The system would approach the equilibrium macrostate smoothly with no noticeable fluctuations
away from equilibrium.
- The system would bounce around randomly among all of the possible macrostates.
An Ideal Gas (in a Box)
Close out the CoinFlipEquilibrium simulation by closing the CoinFlip window. We’ve done enough with our toy
model involving coins. Now let’s look at something a little closer to the real world. We will consider
an ideal gas in a box. An ideal gas consists of molecules that don’t interact with each other in any
way, but simply fly around exhibiting inertial motion until they hit a wall and bounce off. We will
assign each molecule a random initial velocity (with completely random direction and random speed
assigned according to something called the Maxwell-Boltzmann distribution), but we will start with
all of the molecules on the left side of the box at randomly assigned positions. (Note: you may be
wondering how the molecules got to these random positions with these random velocities. One way
to achieve this would be to allow the molecules to interact with each other, which would make this
system chaotic and effectively randomize the positions and velocities.) What will happen when we
remove the barrier in the center of the box and let the molecules move freely from one side to the
other?
In the menu on the left double-click the green arrow next to IdealGasExpansion. This should pop up two
windows. The Ideal Gas in a Box window shows an animation of the particles bouncing around in the box
(note that the left side of the box is red and the right side is blue). The Particle Numbers window
shows a plot of the number of particles on the left (in red) and on the right (in blue) as a function of
time.
- Set the number of particles to 20. Run the simulation until the time reaches about 30, then pause it. Which of
the following statements best describes the results of the simulation?
- Most of the particles stayed on the left side of the box and never went to the right side.
- Most of the particles moved to the right side of the box and then came back to the left side as a
group.
- The particles spread out until there were about equal numbers in each side, but the numbers
showed significant fluctuations about this equilibrium state.
- The particles spread out until there were the same number of each side and then maintained this
equilibrium state without noticeable fluctuations.
- Now we can try to relate the behavior of this ideal gas to the behavior of our coin model. We can
think of each particle of the gas being represented by a coin, with heads indicating the particle is
on the left side of the box and tails indicating it is on the right side. The passage a particle
from one side of the box to the other is then represented by flipping over the corresponding
coin. How do the results of this simulation compare with the results of your 20 coin experiment
in Part I (and the 20 coin simulation above)? How are they similar? How are they different?
- Now set the number of particles to 200. Run the simulation until the time reaches about 50, then pause. How
are these results different from the 20 particle simulation? Comment on the size of the fluctuations relative to
the size of the system. You should try the simulation with 2000 particles to make sure you can see the relevant
trend.
- Consider all three ideal gas simulations. Based on what you learned from the coin model we studied earlier do
you think the entropy of the system generally increased, decreased, or stayed constant during the initial phase
of the simulation? Explain your answer by making an analogy to the results of the coin experiments.
- It is important to recognize that there are some differences between our simple coin model and an ideal gas. In
the coin model we can specify the microstate by stating whether each coin is heads or tails. But to specify the
microstate of our ideal gas we must give the position and velocity of each particle. In the exercises above we
have focused on whether each particle is on the right or left side of the box, but in doing so we have ignored a
great deal of information about the particles in the gas. So our ideal gas doesn’t quite behave the same way as
our row of coins, and therefore the analogy you used to decide about the entropy of the gas is not
exact. In fact, the entropy of an ideal gas depends on the temperature of the gas and the volume
it occupies. At constant temperature (as in our simulations) the entropy of the gas increases
with volume. Does this change your answer to the previous question? Explain why or why not.
- Remember that entropy is really just a measure of the number of microstates in a given macrostate
(S = lnΩ). Which of the following macrostates of our gas in a box do you think has the most
microstates?
- The particles are spread evenly through the left side of the box (with no molecules on the right
side).
- The particles are spread evenly through the left two-thirds of the box.
- The particles are spread evenly through the left three-fourths of the box.
- The particles are spread evenly through the entire box.
- Which of the above macrostates has the fewest microstates?
Entropy and Heating
Now let’s try to see how the increase of entropy connects to another version of the Second Law of Thermodynamics,
namely:
Energy will only flow spontaneously from an object with a higher temperature to an object with
a lower temperature. It will not spontaneously flow in the opposite direction.
We will stick with the ideal gas in a box, but this time we are going to have two gases. We will examine a system that
starts off with a cold gas in the left side of the box and a hot gas (composed of the same type of molecules,
for simplicity) in the right side, with a barrier between the two. Recall that temperature is really a
measure of the average kinetic energy of the molecules in the gas, so the molecules on the right will be
moving slower on average than the molecules in the left side. What happens when we remove the
barrier?
Close the IdealGasExpansion simulation (if it’s still open). Double-click the green arrow next to
HotAndColdIdealGases. The Hot/Cold Gas window shows an animation of the gas, which starts off with the cold
(black) particles on the left and the hot (green) particles on the right. Set the number of green and black particles to
200. Click the Temp Plots box to show a plot of the temperature of the gas in each side of the box
(the temperature of the left side is in red, that of the right side in blue). The temperature of each
side is calculated by first finding the average kinetic energy of all the molecules in that side of the
box.
- Click Start to run the animation and let it run until the time reaches about 50. (Note: you can speed
up the simulation by moving the Speed slider to the right.) What happens to the temperatures of the
two sides?
- Once the two sides of the box reach the same temperature, do they both stay at that temperature or
does the temperature fluctuate?
- When the two sides of the box reach the same temperature, how does that temperature compare to
the starting temperatures of the two sides? Be as precise as possible in your answer.
- Recall your conclusions about the entropy of the ideal gas in the last section. Did the entropy of the hot
gas increase during this simulation? Did the entropy of the cold gas increase? The entropy of the whole
system is just the sum of the entropies of the two gases. So did the entropy of the system increase? In
other words, did the behavior of this system conform to the entropy version of the Second Law?
- Which of the following macrostates of this system has the most microstates?
- All the hot particles are on one side and all the cold particles are on the other.
- All of the particles (hot and cold) are on one side.
- The hot and cold particles are mixed together with about half of each type on each side of the
box.
- Consider the flow of thermal energy in this system. Did the behavior of this system conform to the heating
version of the Second Law? Explain.
- Real gases don’t behave like ideal gases except under certain restrictive conditions (very low density). Instead,
the molecules of real gases interact with each other and exchange energy. If a fast-moving molecule collides
with a slow-moving molecules is it more likely that the fast-moving molecule will give some energy to the
slow-moving molecule, or that the slow-moving molecule will give some energy to the fast-moving
molecule? After MANY such interactions, what will happen to the speed of the molecules in the gas?
- Whether the energy is transfered by the motion of molecules (as in the ideal gas) or by interactions (as in a
real gas), in which situation is the entropy of the gas greatest?
- When the energy is spread evenly among all the molecules and all the molecules are clustered in
a small region of the box.
- When the energy is concentrated in a small number of molecules that are spread evenly throughout
the box.
- When the energy is spread evenly among all the molecules and all the molecules are spread evenly
throughout the box.
- When the energy is concentrated in a small number of molecules which are clustered in a small
region of the box.
Historical Objections to Boltzmann’s Ideas
Boltzmann originally used the motion of molecules to derive the second law in
1872.
This derivation made it appear that the second law is absolute, that the entropy of an isolated system can never
decrease. But as we have seen Boltzmann’s later ideas about entropy (first presented in 1877) indicate
that the second law is only very likely to hold true. In fact, we have seen violations of the second
law in the models we have studied. What made Boltzmann abandon the view that the second law is
absolute?
- As early as 1869 James Clerk Maxwell had expressed some doubts about the absolute nature of the
second law. He imagined a gas contained within a box and monitored by an entity (later known as
Maxwell’s Demon) that would allow only fast moving molecules to move from the right side of the box
to the left side, and only slow moving molecules to move the other way. To see the effects of Maxwell’s
Demon double-click the green arrow next to IdealGasMaxwellsDemon. Set the number of particles
to 200 and show the Temp Plots. Run the simulation for a while, then click Demon On and let the
simulation run for a while longer. Describe what happens to the temperature of the two sides of the
box after you turn the Demon on.
- Does the behavior of the gas appear to violate the Second Law? Explain how you can tell.
- Consider this: does the gas in this simulation really constitute an isolated system? In other words, does
it interact with anything other than the box? If so, what?
- For a full accounting of the change in entropy for this system we would need to keep track of the
entropy of the Demon itself, which we don’t know how to do. But Maxwell’s concern was that the
natural motion of the gas molecules might mimic the effects of the Demon. While it is hard to argue
that this is impossible, do you think that the kind of motion shown in this simulation is likely to occur
(without the intervention of Maxwell’s Demon)?
- Now turn off the Demon (uncheck the Demon On box) and let the simulation continue to run. Describe
what happens to the temperature of the two sides of the box after the Demon is turned off. After you
finish this you can quit the Maxwell’s Demon simulation.
- Another objection to Boltzmann’s H-theorem was presented by Josef Loschmidt in 1876. He pointed
out that the law of Newtonian mechanics are time reversible, which means that they work equally well
backwards in time as they do forwards. If a certain set of motions leads to an increase in entropy,
then the time-reversed motion will lead to a decrease in entropy. Both sets of motions are allowed by
Newton’s Laws. One way to produce these time-reversed motions is to reverse the direction of motion
of each molecule. The motion of the system after the reversal will be a time-reversed version of the
motion before the reversal. To visualize this effect run the IdealGasExpansion simulation again. Let
the simulation run until the gas reaches equilibrium, then hit the Reverse button. Let the simulation
run for longer than you let it run before hitting Reverse. Describe what happens to the gas.
- After the velocities are reversed, what happens to the entropy of the gas? Does this constitute a genuine
violation of the Second Law?
- Close the IdealGasExpansion simulation and run the HotAndColdIdealGases simulation again. Use 200
particles of each type and show the Temp Plots. Run the simulation and let it go until time reaches
about 20, then hit Reverse. Watch the simulation for a while. What happens to the temperature of the
two sides of the box after the velocities are reversed? Does this constitute a violation of the Second
Law?
- These objections led Boltzmann to reformulate his interpretation of the Second Law. In 1877 he
presented the statistical approach to entropy that we have been examining. He admitted that violations
of the second law are possible but argued that such violations would be highly unlikely and would be
short-lived in any macroscopic system. Based on what you have seen in the coin flip model and the
computer simulations, would you agree that violations of the Second Law are unlikely to occur in a
gas with 1024 molecules? Are such violations possible?
- Consider what a radical shift this is in the concept of a “law of nature.” Are we really justified in
calling the Second Law of Thermodynamics a “Law” if it is just a statement of probability? Defend
your answer.
The Scope of the Second Law
Like all physical theories the Second Law of Thermodynamics has limited scope. Let’s explore that scope a little
bit.
- In our coin experiment we found that with 20 coins (and to a lesser extent even with 200 coins) we got
occasional large fluctuations in the number of heads that resulted in a decrease in the entropy of the
system. Boltzmann argued that noticeable fluctuations would essentially never occur in a gas of 1024
molecules. What does this example tell us about the scope of the Second Law?
- We have seen that a gas interacting with Maxwell’s Demon can appear to violate the Second Law if we
ignore the entropy of the demon itself. Use this example to discuss why we must restrict the Second
Law to isolated systems. Why is it that the entropy of a system can decrease if that system is not
isolated? If we expand the system to include everything that is interacting with the original system
(possibly to include the entire Universe), could the entropy of this expanded system decrease?
- Evolution of life on Earth shows a clear tendency to produce more organized (ordered) structures over
time. Does this violate the Second Law? If the entropy of all living things on Earth is decreasing over
time, what else must be happening according to the Second Law?