Working papers

Deep Reinforcement Learning and Macroeconomic Modelling (PhD Thesis)

Current version:  2024 

• Chapter 2 was circulated under the name "Deep Reinforcement Learning: Emerging Trends in Macroeconomics and Future Prospects" as IMF Working Paper No. 2022/259. 
• Chapter 3  was circulated under the name "Learning from Zero: How to Make Consumption-Saving Decisions in a Stochastic Environment with an AI Algorithm" as CESifo Working Paper No. 9255.
• Chapter 4 was circulated under the name "Can an AI agent hit a moving target?".

Abstract:

I introduce a theory of bounded rationality, utilizing an AI framework to account for the inherent limitations and characteristics in human decision-making processes. The key focus of my contribution lies in implementing this framework within general equilibrium models to highlight the adaptive capabilities of AI agents. Additionally, I take into consideration potential genetic disparities among human actors in the decision-making process by integrating an exploration mechanism. Across three chapters, I examine the integration of AI representative agents in well-established general equilibrium models in macroeconomics.

Chapter 2 serves as an introductory section, delving into the methodological frame- work’s historical development and theoretical foundations. I explore the applications of deep RL in economics, critically analyzing the framework’s potential benefits and limitations when employed in macroeconomics.

In Chapters 3 and 4, I present the simulation experiments conducted in this thesis, aiming to assess the usefulness of the proposed AI framework in modelling adaptive behaviors. The first application focuses on a stochastic optimal growth environment with dynamic changes in income processes over time. The second application centers around a transaction cost of money demand model, involving a shift in the monetary policy regime from a constant to an increasing nominal money supply.

Through these simulation experiments, I demonstrate the versatility and advan- tages of the deep RL framework to model economic agents. Additionally, the results highlight the significant influence of exploration variations on distinct adaptive be- haviors. Moreover, I discuss the limitations of the deep RL framework.

Population Aging in ASEAN+3: But is 60 the New 40?

with Hangman Zhao 

Current version: AMRO Working Paper No. WP24-08 July  2024

Abstract:

Population aging is becoming a significant concern, particularly as its pace accelerates, especially in emerging market economies. However, labeling all individuals aged 65 and above as elderly can be misleading and inaccurate when life expectancy is increasing. Therefore, using the prospective old-age dependency ratio to define what is elderly would allow for more precise measurements and facilitate research into the impact of aging on economic growth. Our findings suggest that while a negative relationship between aging and economic growth at the global level was more prominent before 1990, this negative effect has decreased over time. Moreover, the population nearing retirement age exhibits an increasing contribution to growth. Harnessing the potential of those typically deemed old by traditional measures, yet who remain productive, could effectively bolster economic development. Additionally, we find that the impact of aging on growth varies across individual economies in the ASEAN+3 region. The accumulation of human capital and technological advancements appears to mitigate negative effect from aging, underscoring the need for economies to promote both as their populations age.

AI and Macroeconomics Modelling: Deep Reinforcement Learning in an RBC Model

with Tohid Atashbar 

Current version: IMF Working Paper No. 2023/040 February 2023

Abstract:

This study seeks to construct a basic reinforcement learning-based AI-macroeconomic simulator. We use a deep RL (DRL) approach (DDPG) in an RBC macroeconomic model. We set up two learning scenarios, one of which is deterministic without the technological shock and the other is stochastic. The objective of the deterministic environment is to compare the learning agent's behavior to a deterministic steady-state scenario. We demonstrate that in both deterministic and stochastic scenarios, the agent's choices are close to their optimal value. We also present cases of unstable learning behaviours. This AI-macro model may be enhanced in future research by adding additional variables or sectors to the model or by incorporating different DRL algorithms.

Deep Reinforcement Learning: Emerging Trends in Macroeconomics and Future Prospects

with Tohid Atashbar 

Current version: IMF Working Paper No. 2022/259 December 2022

Abstract:

The application of Deep Reinforcement Learning (DRL) in economics has been an area of active research in recent years. A number of recent works have shown how deep reinforcement learning can be used to study a variety of economic problems, including optimal policy-making, game theory, and bounded rationality. In this paper, after a theoretical introduction to deep reinforcement learning and various DRL algorithms, we provide an overview of the literature on deep reinforcement learning in economics, with a focus on the main applications of deep reinforcement learning in macromodeling. Then, we analyze the potentials and limitations of deep reinforcement learning in macroeconomics and identify a number of issues that need to be addressed in order for deep reinforcement learning to be more widely used in macro modeling.

Deep Reinforcement Learning in a Monetary Model

joint with Mingli Chen, Andreas Joseph, Michael Kumhof, Xinlei Pan,  and Xuan Zhou.

April 2021. Rebuilding Macroeconomics Working Paper Series No.58