A Bridge between kinetics and information theory?

Observations, Energy, Action, and Entropy

Entropy as we know it is a property that stays constant over time for a given macrostate of a given system. It represents something like the volume of available microstate space per macrostate. What is a macrostate? It’s essentially a label we decide on based on what we’re looking at in our model. For instance, if we’re measuring pressure, then we can imagine assigning a certain arrangement of velocities to the various gas particles of a system. There are many ways to assign those velocities which give the same pressure at a given temperature. Some pressures have fewer such arrangements than others for a given temperature, and thus are less likely to occur. Such states are higher energy according to the Boltzmann relation.

This gives us the classical Boltzmann entropy.

\(S = k_B \ln \Omega \)

But there’s something missing here. At a given time, that whole volume of state space is not available in the next time step. Particles must transition into adjacent states, or at least they are heavily constrained by an increased probability of doing so. This means that at a given moment, only a fraction of the total state volume is accessible. We can define that as something like the below:

\(S_{\text{acc}}(t) = -k_B \sum_{x \in \Gamma(t)} p(x) \sum_{y \in \Gamma(t+1)} P(x \to y) \ln P(x \to y) \)

Here we are saying that our entropy is the weighted sum of the log probability of each transition. Now instead of viewing entropy as a fully time-independent quantity, we can consider the contribution at a given time to a system’s entropy over a certain period.

For our gas in a box model, this is constant. From a given state at time step one, consider the number of states that could be transitioned to at time step 2. Ultimately the size of that space is the same as the space of possible states at time step 1. When you sum all of the possible adjacent states of all the available states it’s very often equivalent to the total size of the state space.

But we can imagine situations where it isn’t. There are contexts where once a state is visited, the probability of re-visiting it is much lower. Take a ratchet for instance. And in these cases, rather than consider entropy to be the probability of each state, we can consider entropy instead to be based instead on some aggregation of the conditional probabilities of entering a given state given its probability of existing in current states. In doing this we arrive at the foundation of a form of path entropy.

When microstates are removed from the system, we’ve discussed this as constraint, which we’ve discussed as a form of potential energy.

When they are added, there is necessarily an increase in energy of the system per the Boltzmann relationship if the value of all macrostates are held constant. This is because the average probability of the system entering any individual state must decrease. Hence the average energy increases according to the Boltzmann relation.

If we’re assuming constant temperature and pressure, then the added states would increase the kinetic energy of the system. That’s because these states are getting added to the phase space, meaning they’re either added as new values of momentum that can be taken, or new values of position. If we increase the range of available velocities for the particles, that results in an increased kinetic energy, as the velocities either get more negative or more positive. Otherwise, we have to increase the volume in which case to keep the same pressure, we have to increase the velocity of the particles to cover the same area and create the same number of collisions (i.e. kinetic energy).

Let’s bring this to the concept of action. We’ve defined the addition or removal of states as kinetic or potential energy respectively. If we take the number of microstates added (kinetic energy), and subtract the number removed (potential energy), we have our Lagrangian. When we take the integral of this relationship over time, we arrive at the action. We also arrive at the total difference in available states for the system to occupy. The principle of least action suggests that this quantity is minimized. So a restatement of that principle in our terms is that the system takes a trajectory which minimizes the number of additional states added to the system.

\(L_{\text{info}} = \underbrace{m \dot{x}^2}_{\text{microstates added}} - \underbrace{k x^2}_{\text{microstates constrained}} \)

This absolutely makes sense. Why? Because we as human reject models which add unnecessary terms and spaces. We seek models which minimize phase space, and hold the minimal ones as the most true or fundamental.

We should now have a basic way to use information to describe kinetics. We can apply this idea to a simple harmonic spring. When the spring is compressed or stretched, we’re storing potential energy. Constraint is occurring. Rather than be able to move to the right or left, the mass is heavily biased to move in one direction. This essentially removes half the phase volume for the mass. As described previously, this is the creation of a form of symmetry. Would-be trajectories on the collapsed half of the phase volume are redirected or reduced in traversed volume. This is a form of symmetry. We can also consider it a form of symmetry that the time spent at that particular point or sufficiently nearby is much longer than the time spent in the middle. Therefore there are many phases at which an oscillation can occur and be found at that particular point within a certain time and distance range. Since the mass will tend to go slowly at the ends, it will tend to linger at that point. In the middle, the mass never lingers. There is a lower probability for it to be found at that point at a given time, so it further refines the set of phases when compared to the mass being found at the maximum compression position. I’m not as strong as I’d like to be on the math, but I think in the terms that this video lays out, we would consider the different ways to arrive at the same position as gauges for the purposes of calculating action.

Let’s then take a ratchet example. In this case the set of available states changes. when the ratchet is moving in the free direction, at any given moment, there is a slight gradient against its forward motion. In this case the mechanism is largely free to rotate in either direction. Therefore work which is put into its handle is translated into kinetic energy. There is a direct increase in the available velocities, and there is also more asymmetry in any given state of the system. Based on the angular velocity and angular position of the ratchet, as well as corresponding starting information, we can recover some information about the relationship between the time and force with which the handle was pushed. Now let’s go the other way. If we suddenly change and push the ratchet in its bound direction, we push against something. Let’s first imagine that thing doesn’t give. We push against it and it is stuck. The handle is in one place. The system has maximum potential energy. It also has maximum symmetry, or reduced phase volume. I can put many forces at many times and end up at the same position. As soon as the bolt loosens, the phase volume increases again, the potential energy decreases, and the kinetic energy increases, even if the force imparted on the handle was the same.

One way to remove states in the state space is through observations. An observation produces a temporary reduction of available state. This happens at a single point in time and the available state space can quickly recover to the level of the un-observed system. Each of these represents essentially a restriction of phase space. If we observe the exact state of a system for a moment, we have done the equivalent of reducing available phase space. But this doesn’t happen permanently. Quickly, the uncertainty about which trajectory the system took causes the available phase volume to include all microstates of the system. Quickly, but not immediately. If one were able too sample the state of the system with enough frequency, it would do something like the equivalent of constraining the system in one state, even if the states were allowed to change. Bringing this back to mutual constraint and energy: the result would be potential energy.

In other words, observations create an impulse of potential energy over a certain period of time, which dissipates if not maintained. In order to maintain the potential energy, the observations must be repeated. The rate at which the observations are made determines the amount of potential energy possible to maintain.

Increasing the rate of observations reduces the value of the action integral. We also know that observations sustained over time create information. If we continuously observe which half of the x axis an object is on, we’ve created 1 bit of information.

The maintenance of that information requires sustained energy. That would mean the instantaneous quantity which is produced in the course of the observation has Planck units.