Consider the problem of a government that wants to reduce the debt-to-GDP (gross domestic product) ratio of a country. The government aims at choosing a debt reduction policy which minimises the total expected cost of having debt, plus the total expected cost of interventions on the debt ratio. We model this problem as a singular stochastic control problem over an infinite time-horizon. In a general not necessarily Markovian framework, we first show by probabilistic arguments that the optimal debt reduction policy can be expressed in terms of the optimal stopping rule of an auxiliary optimal stopping problem. We then exploit such link to characterise the optimal control in a two-dimensional Markovian setting in which the state variables are the level of the debt-to-GDP ratio and the current inflation rate of the country. The latter follows uncontrolled Ornstein-Uhlenbeck dynamics and affects the growth rate of the debt ratio. We show that it is optimal for the government to adopt a policy that keeps the debt-to-GDP ratio under an inflation-dependent ceiling. This curve is given in terms of the solution of a nonlinear integral equation arising in the study of a fully two-dimensional optimal stopping problem.