Entropy

A general definition of entropy was formulated by Boltzmann in 1872. It is expressed in terms of "coarse-graining volume" in the phase space, which amalgamates the positions and momenta of all particles in a system into one point (Figure 01c). The relentless increase toward higher entropy until reaching its maximum (i.e., in a state of thermal equilibrium) is related to the fact that the evolution of the phase point is more favorable toward the larger "coarse-graining volume"

• Configuration Space - It is a space consists of all the 3-dimensional spatial coordinates of N particles (N = 4 in Figure 01c, represented by the blue arrows) with all the 3N coordinate axes orthogonal (perpendicular) to each others. The horizontal axis for the phase space in Figure 01c is a much simplified visual aid for the 3N configuration space. At 300^oK and standard atmospheric pressure of 101 kpa, the number of gas molecules N in a cube of 10 cm would be about 3x10²².
• Momentum Space - In addition to the position of the particle, each one needs at least three more numbers to specify its state, namely the three components of its momentum (red arrow in Figure 01c). Similar to the configuration space, the momentum space is made up by 3N orthogonal axes representing the momenta of N particles. At 300^oK and standard atmospheric pressure of 101 kpa and assuming the gas molecules to be hydrogen atoms with mass m = 1.67x10^-24gm, the root-mean-square velocity of the particles v_rms = (3kT/m)^1/2 ~ 3x10⁴ cm/sec., the corresponding momentum p = mv_rms = (2mE)^1/2 ~ 4.5x10^-20 erg-sec/cm (or E ~ 10^-16 erg ~ 6x10^-5 ev). The size of the momentum space for each particle can be estimated from a range below and above the rms value such that about 0.1% probabilities toward the tail ends are excluded.
• Phase Space - It is the orthogonal combination of the configuration and momentum spaces having altogether 6N dimensions as shown in Figure 01c. The dimensions are often referred to as the degrees of freedom. The phase space volume W is:
  
  W = {[^3N/2(2mE)^3N/2V^N]/[(N!(3N/2)]}(E/E), where
  
  p = (2mE)^1/2(E/2E) is the range of momentum,
  2^3N/2(2mE)^(3N-1)/2 comes from integrating up to the energy E = p²/2m,
  V is the spatial volume containing the particles,
  N! and (3N/2) are for removing the degeneracy related to the permutation symmetry of identical particles. (3N/2) is the Gamma function identical to (3N/2)! if the argument is an integer.
• Partition Function - It is the the number of microscopic states within the energy shell E of the phase space. The Planck's constant h = 6.625x10^-27 erg-sec from the uncertainty relation px ~ h in quantum theory is conveniently taken as the basic unit (minimum size) of the microscopic states. Thus the partition function Z is just:
  
  Z = W/h^3N = {[(^1/2(2mE)^1/2V^1/3)/h]^3N/[N!(3N/2)]}(E/E) ~ {(10⁸)^3N/[N!(3N/2)]}(E/E)
  
  where the numerical value 10⁸ is computed from the previous assumptions for the size of the container and p. It shows that the number of microscopic states available is enormous in the order of 10²⁴ even for a system of just one particle (N = 1).
  
• Entropy - Boltzmann's definition of entropy S is:
  
  S = k ln(Z)
  
  where k = 1.38x10^-16 erg/^oK is the Boltzmann constant. It is immediately clear that entropy would increase by adding # of particles N, energy E, or volume V as shown in figure 01b (the internal degrees of freedom are not considered here). Since Z depends on the parameters in power of 3N, it varies by huge amount with a relatively small change in these parameters.
• Coarse-graining Region - Each of this sub-volume w in the phase space is characterized by some macroscopic properties such as temperature, pressure, density, color, chemical composition etc. with a certain number of microscopic states. The w sub-volume has number of neighbors going up drastically with increasing dimension - typically 6 in the 2 dimensional case, 14 in 3 dimensions, ... As mentioned above, the various w sub-volumes tend to differ in size by absolutely enormous factors.
• Second Law of Thermodynamics - The evolutionary path of a phase point in the phase space is indicated by a curve as shown in Figure 01d. Although time and hence rate of change is absent in the picture, the direction of evolution is represented by an arrow. The path is determined by physical law such as the N-body Newtonian equation of motion, it has a higher probability of moving into another w sub-volume with larger size and hence higher entropy - the basic conception of the Second Law of Thermodynamics. The appearance of randomness is the manifestation of the fact that there are so many different microscopic states available for the same macroscopic state. The system reaches thermal equilibrium when the phase point enters the largest sub-volume and keeps wandering around inside. Note that there is a certain probability of going into a smaller w, but the probability goes down rapidly with decreasing sub-volume size.