Reading Group Syllabus
Below you will find the syllabus for the reading group, which occurs every Thursday at 11AM through November 18, 2021.
Session 1 (8/19/2021): Introduction to Model and Systems Thinking (pt. 1)
Readings
- Principles of modeling and simulation; Chapter 1
- A tutorial on conceptual modeling for simulation; Sections 1-3
Learning Objectives
- Define and describe modelling
- Define and describe simulation
- Describe the advantages and disadvantages of modelling and simulation (M & S)
- Describe conceptual modelling
Discussion Questions
- What is modelling?
- What is simulation?
- How are the M & S distinct from other analytic tools?
- What is conceptual modelling?
Session 2 (8/26/2021): Introduction to model and systems thinking (pt. 2)
Readings
- Principles of modeling and simulation; Chapter 2
- A tutorial on conceptual modeling for simulation; Sections 4-8
Learning Objectives
- Describe how M & S can be used to solve analytic problems
- Describe how M & S can be used to gain insight
- Describe the role of uncertainty in M & S
- Describe key requirements of a good conceptual model
Discussion Questions
- How can M & S be used to solved analytic problems?
- How can M & S be used to gain insight?
- What is the role of uncertainty in M & S?
- What are key requirements of a good conceptual model?
Session 3 (9/2/2021): Ethics
Readings
- Bak, M.A.R. (2020) Computing Fairness: Ethics of Modeling and Simulation in Public Health
- Recommended: Shults FL, Wildman WJ, Dignum V. The ethics of computer modeling and simulation. In 2018 Winter Simulation Conference (WSC)
Learning Objectives
- Describe the various ways simulation designs express ethical assumptions
- Describe how to use models and simulations responsibly and ethically
- Describe how to represent norms within simulatinos and why this matters
- Describe how to consult with stakeholders about the ethical dimensions of designing a simulation
Discussion Questions
- How do simulation designs express ethical assumptions?
- How can simulations be used responsibly and ethically?
Session 4 (9/9/2021): Introduction to Modelling Methods and Elements (pt. 1)
Readings
- Marshall DA, Burgos-Liz L, IJzerman MJ, Osgood ND, Padula WV, Higashi MK, Wong PK, Pasupathy KS, Crown WApplying dynamic simulation modeling methods in health care delivery research—the SIMULATE checklist: report of the ISPOR simulation modeling emerging good practices task force.Value in health. 2015 Jan 1;18(1):5-16
Learning Objectives
- Describe Agent Based Models (ABMs), Discrete Event Models (DEMs), and Systems Dynamics Models (SDMs)
- Compare and contrast features of ABMs, DEMs, and SDMs
- Describe the SIMULATE checklist for assessing whether simulations are appropriate for addressing a given problem
Discussion Questions
- What are disinct features of ABMs, DEMs, and SDMs?
- What are the similarities and differences of ABMs, DEMs, and SDMs?
- What are SIMULATE criteria for assessing whether M & S are appropriate for addressing a given problem?
Model Example: Interaction
Session 5 (9/16/2021): Introduction to Modelling Methods and Elements (pt. 2)
- Marshall DA, Burgos-Liz L, IJzerman MJ, Crown W, Padula WV, Wong PK, Pasupathy KS, Higashi MK, Osgood ND. Selecting a dynamic simulation modeling method for health care delivery research—Part 2: Report of the ISPOR Dynamic Simulation Modeling Emerging Good Practices Task Force.Value in health. 2015 Mar 1;18(2):147-60.
- Recommended: Robinson S. A tutorial on conceptual modeling for simulation. In 2015 Winter Simulation Conference (WSC) 2015 Dec 6 (pp. 1820-1834). IEEE.
- Recommended: Spiegel M, Reynolds PF, Brogan DC. A case study of model context for simulation composability and reusability. In Proceedings of the Winter Simulation Conference, 2005. 2005 Dec 4 (pp. 8-pp). IEEE.
Learning Objectives
- Identify which simulation modelling method is most appropriate for a given purpose and object
- Describe fundamental elements of good practices in modelling
- Describe differences between dynamic and mode traditional simulation modelling method
Discussion Questions
- What criteria can be used for deciding which simulation modelling method if best for a given purpose and objective?
- What are fundamental elements of good practices in modelling?
- What are differences between dynamic and more traditional simulation methods?
Model Example: Emergence
Session 6 (9/23/2021): Modelling Considerations and Best Practices for ABMs
Readings
- Hammond RA. Considerations and best practices in agent-based modeling to inform policy. In Assessing the use of agent-based models for tobacco regulation 2015 Jul 17. National Academies Press (US).
Learning Objectives
- Describe common uses of ABMs
- Describe the key building blocks of ABMs, using the Properties Actions Rules Time and Environment (PARTE) framework
- Describe best practices for development of ABMs
Discussion Questions
- What are common uses of ABMs?
- What are the building blocks of ABMs?
- What are ABM development best practices?
Session 7 (9/30/2021): ABMs (Emergence)
Readings
- Friesen, M. R., Gordon, R., & McLeod, R. D. (2014). Exploring Emergence within Social Systems with Agent Based Models. In D. Adamatti, G. Dimuro, & H. Coelho (Ed.), Interdisciplinary Applications of Agent-Based Social Simulation and Modeling (pp. 52-71). IGI Global.
Learning Objectives
- Describe what ABMs are able to model and what scenarios should not be modelled using ABMs
- Describe assumptions inherent in modelling with ABMs
- Define and describe the concept of emergence
- Define and describe the different modelling units (e.g., agents, properties and interactions) in ABMs
Discussion Questions
- Do ABMs seem to require collaborative model building? Who are the people who need to be involved in the model building process?
- Does your area of research/interest seem like a good fit for ABM? Why?
Session 8 (10/7/2021): Case Study--Infectious Disease in Bogota
Readings
- Gomez J, Prieto J, Leon E, Rodrıguez A (2021) INFEKTA—An agent-based model for transmission of infectious diseases: The COVID-19 case in Bogota, Colombia. PLoS ONE 16(2):e0245787.
Learning Objectives
- Describe and contrast compartmental models and ABMs as approaches to COVID-19.
- Match the different components of the model to the modeling units and interactions discussed in last week’s reading group on ABMs
Discussion Questions
- Do the assumptions in the model seem reasonable?
- Is there anything missing from this model that you think should have been included or vice versa?
- How could this model be adjusted to be applicable globally?
- What are the strengths and weaknesses of using ABMs to model COVID-19? How do compartmental models or complex systems take account of these limitations?
Model Example: ABM
Session 9 (10/14/2021): DEMS
Readings
- Wainer, G. A. (2009). Discrete-event modeling and simulation: a practitioner’s approach. Boca Raton, CRC Press.
Learning Objectives
- Describe which scenarios are appropriate for discrete event models and which scenarios should not be modeled using discrete event models
- Identify assumptions inherent in modeling with discrete event models
- Identify the different modeling units inherent in discrete event models
- Describe how discrete event models differ from ABMs
Discussion Questions
- Does this type of model seem like a good fit for your research?
- When thinking about learning a new type of modeling and translating the process to applied researchers, would you prefer to explain discrete event models or ABMs? Why?
Session 10 (10/21/2021): SDMs
Readings
- Elsawah, S., Pierce, S.A, Hamilton, Serena H, Van Delden, H., Haase, D., Elmahdi, A., and Jakeman, A.J., “An overview of the system dynamics process for integrated modelling of socio-ecological systems: Lessons on Good Modelling Practice from Five Case Studies.” Environmental Modelling & Software: With Environment Data News 93 (2017): 127-45.
Learning Questions
- Describe what SDMs models are able to model and what scenarios should not be modeled using SDMs
- Identify assumptions inherent in SDMs
- Describe how space can be modeled in SDMs Identify how SDMs differ from DEMs and ABMs
Discussion Questions
- If you have taken courses/worked with simulation modeling, did you learn about this?
- Is it the modelers responsibility to explain how the modeling process may contribute?
- Out of the three modelling modalities we have learned about, which one relies more on the programmer and which on Subject Matter Experts (SMEs)? Does this make one better than the others in your mind?
Model Example: Combining ABM, DEM, SDM Simulation Modalities
Session 11 (10/28/2021): Case Study--Planning Beyond the Worker
Readings
- Peck, W.J., and Finke, D.A., “Systems_Dynamic_Modeling_Planning_Beyond_the_Worker.” 2019 Winter Simulation Conference (WSC), 2019, 2606-616.
Learning Objectives
- Describe how SDM is effective in estimating long-run resource requirements
- Identify the basics of SDM to observe the nuances of creating a model
Discussion Questions
- What are some basic steps of creating an SDM?
- How could you alter the process using different real-world data?
- How could this case study be incorporated into your research and modeling?
Session 12 (11/4/2021): Parameterization and Calibration 1
Readings
- Hunt, R.J., Doherty, J., and Tonkin, M.J., “Are Models Too Simple_ Arguments for Increased Parameterization _ Enhanced Reader.” Ground Water 45, no. 3 (2007): 254-62.
Learning Objectives
- Identify and describe traditional model calibration approaches
- Identify and describe the regularized inversion calibration approach
- Identify differences between the two calibration approaches
Discussion Questions
- Should models be as simple as possible?
- How have you used the traditional model calibration and regularized inversion approach in your modeling?
Session 13 (11/11/2021): Parameterization and Calibration 2
Readings
- Qiu, N., Park, C., Gao, Y., Fang, J., Sun, G., and Kim, N. H. (November 9, 2017). Sensitivity-Based Parameter Calibration and Model Validation Under Model Error ASME. J. Mech. Des. January 2018; 140(1): 011403.
Learning Objectives
- Identify discrepancy terms to avoid biased calibration parameters
- Describe a simple calibration based on sensitivity information resulting from model errors
- Identify the differences between sensitivity-based calibration, conventional least squares calibration, and Bayesian calibration method
Discussion Questions
- How can sensitivity-based calibration be applied across disciplines?
- What are the strengths and weaknesses of SBC?
- What high-dimension design problems are you working on that can benefit from SBC?
Session 14 (11/18/2021): Verification and Validation
Readings
- Gore RJ, Lynch CJ, Kavak H. Applying statistical debugging for enhanced trace validation of agent-based models. Simulation. 2017 Apr;93(4):273-84.
Learning Objectives
- Define verification and validation
- Identify the characteristics of trace validation
- Identify the benefits of the V&V calculator
- Identify and describe the verification and validation challenges of ABMs
Discussion Questions
- What are differences between verification and validation?
- What is trace validation and the V&V calculator?
- What experimental evaluations have you developed that could benefit from trace validation?
- How could trace validation be strengthened to be implemented to ABM more universally?