Explainable AI is a growing subfield in Machine Learning (ML) that seeks to solve this problem by applying various methods to understand the inner workings of black-box Artificial Intelligence agents. Past work at Oregon State University investigated a method of combatting the black-box nature of RNNs by inserting an autoencoder (called a Quantized Bottleneck Network, or QBN) layer in between two layers of an RNN. This QBN encodes the continuous output of the first layer into a discrete (1s or 0s) fixed-length latent representation, and then decodes the original information out of this latent representation. In other words, QBN insertion is a method used to extract finite representations of continuous neural network policies trained with RL algorithms. It is a way of making otherwise black-box neural networks more interpretable. QBN insertion has been applied before to various Atari video games, but not to locomotion tasks with continuous action spaces like controlling motor torques (as opposed to choosing one of some number of finite actions). This project attempts to insert QBN networks into policy networks and extract a finite-state representation of these policies to gain greater explainability.