Mean-field games (MFGs) facilitate otherwise intractable learning in game-theoretical equi- librium problems with many agents. The general approach is to analyze agents via their distribution, which allows to abstract multi-agent stochastic dynamical systems into a single representative agent and the mass of all other agents. We begin by focusing on discrete-time models. We show that fixed point iteration is insufficient for solving MFGs in general. We then present some algorithms based on entropy regularization, dynamic programming and reinforcement learning. In the second half of the talk, we in- corporate graph structure into the model via graphon limits. A numerical reduction of graphon MFGs to standard MFGs allows application of algorithms for general MFGs. Accordingly, we demonstrate intuitive numerical results for exemplary investment and epidemics control problems. Lastly, we briefly touch upon some extensions to other types of graphs. Overall, we obtain a scalable framework for solving large-scale dynamic game-theoretic problems.