What the exercise exam asks you to produce here: the V table after every sweep k with the greedy policy read off it (archetypes A18 and A20), or the value of one fixed policy with the action held at pi(s) and no max (A19), plus the reasoning about gamma that decides which policy wins.
Part 9 chapter · pack index · every number on this page was machine checked against the printed sources
Built for the FAI pack. Traces are precomputed at load and on every parameter change; stepping renders one immutable snapshot, so back and forward are exact inverses. The value iteration, policy iteration and policy evaluation traces in this file were diffed state by state against an independent Python implementation on twelve parameter settings, and every printed number cited above was checked against its source.