Discussion
Through formal assessment of the Macrosystems EDDIE ecological
forecasting curriculum for undergraduates, we found that modules were
successful in increasing student confidence and knowledge of ecological
forecasting and data science (Fig. 2) and lowered the barrier of entry
to these fields for instructors (Fig. 3). In an era when data science
and ecological forecasting skills are increasingly needed to tackle
pressing biological and environmental science problems (Hampton et al.
2017, National Academies of Sciences 2018, Feng et al. 2020, Emery et
al. 2021), the Macrosystems EDDIE curriculum provides one pathway to
introducing these skills to both students and instructors.
Our results indicate that flexible, short, and easy-to-use modules
increase student confidence in data science and ecological forecasting
skills. In particular, students showed the greatest gains in confidence
in ecological forecasting skills (Fig. 2a), likely because they had
lower initial confidence in ecological forecasting skills (e.g.,
generating forecasts, for which students reported a median pre-module
Likert score of 2, or ‘slightly confident’). In comparison, student
confidence in their data science skills was relatively higher prior to
completing the module (e.g., graphing data, with a median pre-module
score of 4, or ‘very confident’; Fig. S1). The Dunning-Kruger effect
(Kruger and Dunning 1999) may explain the few students that exhibited
decreases in confidence (ranging from n = 24 students for the skill of
generating a forecast to n = 77 students for the skill of graphing
data), in which novice students overestimate their abilities, and as
they progress, are much better able to estimate their abilities, which
are less than they previously thought (Fig. S1). Ultimately, increased
student confidence and knowledge of data science and forecasting are
relevant beyond the life sciences, as workers with data science and
predictive modeling skills are sought across multiple sectors (Stanton
and Stanton 2019).
Instructor feedback after teaching a module indicates that the
Macrosystems EDDIE approach of “just-in-time” background skills
training (sensu Novak et al. 1999) and robust instructional
supporting material may be successful strategies for instructor
professional development in data science. We received positive feedback
regarding the effect of Macrosystems EDDIE modules on both the growth of
instructor pedagogical (e.g., active learning) and disciplinary (e.g.,
data science and ecological forecasting) knowledge (Auerbach and Andrews
2018, Andrews et al. 2019). Most instructors said that Macrosystems
EDDIE modules were easy to use and very to extremely effective in
teaching ecological forecasting and data science concepts (Fig. 3).
Qualitative responses to our instructor survey indicated that a
comprehensive introduction to the structure, development, and
interpretation of the forecasting models used in each module (e.g.,
reviewing the structure of a simple ecosystem primary productivity model
in the Intro to Forecasting module) was helpful to both students
and instructors (Text S2). In addition, instructors reported that the
accompanying instructor manual with detailed talking points for each
slide in the introductory presentation and suggested timing for each
activity within the module were helpful for classroom implementation.
Finally, most instructors reported that they were better equipped to use
long-term and high-frequency data and more likely to use sensor network
data after teaching a module (Fig. 3b), indicating that the modules
build skills and data science familiarity with instructors as well as
students. Overall, an important achievement of this adaptable,
accessible curriculum is “training the trainers,” in which an
instructor gains skills and knowledge in a new area, which are then
transferred to students (Beyer et al. 2009, Emery et al. 2021).
Modules were iteratively revised in response to student and faculty
feedback. For example, we revised early versions of the modules to
provide a more in-depth introduction in Activity A to the modeling
approaches used for forecasting as a method of “just-in-time” training
for both students and instructors. In addition, RMarkdown versions of
the Forecasts & Uncertainty and Forecasts & Data modules
were developed based on requests from instructors. The RMarkdown files
provide scaffolding for both students and instructors, who can start by
working through materials in the point-and-click R Shiny interface and
then move to the code “under the hood” of the Shiny application if
they wish. Importantly, this scaffolding may enable students and
instructors to transfer skills learned from teaching the module to their
own research projects, as they can modify the code for their own
datasets and research questions.
Macrosystems EDDIE ecological forecasting modules may facilitate the use
and analysis of large datasets, including NEON data, by instructors who
have not had extensive data science training. While interdisciplinary
collaborations with, e.g., computer scientists can facilitate analyses
with large computational demands, ecologists must still possess basic
data science skills, such as coding and data wrangling, modeling, and
visualization, to make these collaborations a success (Cheruvelil et al.
2014, Cheruvelil and Soranno 2018, Carey et al. 2019). In sum, we found
that the development of comprehensive supporting materials aimed to
provide background skills and pedagogical training for instructors is
critical for the effective implementation of new data science material
into existing undergraduate curricula and may also facilitate new
research efforts for instructors. Up-to-date versions of the modules are
available on GitHub (https://github.com/MacrosystemsEDDIE) and feedback
on module content and ease of use is welcome and can be submitted at
MacrosystemsEDDIE.org.
To train ecological and environmental scientists in data science and
ecological forecasting concepts and skills, these topics need to be
presented in a relevant, approachable way for both students and
instructors. Our data indicate that the Macrosystems EDDIE approach is
effective in engaging both instructors and students in data science and
ecological forecasting, and our observed increases in student confidence
may foster greater student “science identity” and retention in STEM
(Stets et al. 2017, Vincent-Ruz and Schunn 2018, O’Brien et al. 2020,
Bowser and Cid 2021). Ultimately, increased data science confidence and
proficiency by both undergraduate students and instructors unleashes
tremendous potential to leverage large datasets for addressing
environmental challenges.
Acknowledgements
We thank the students and instructors who tested Macrosystems EDDIE
ecological forecasting modules, especially Kait Farrell, Matt Hipsey,
Leah Johnson, Nick Record, and Kiyoko Yokota. We also thank the Virginia
Tech Reservoir Group and our undergraduate focus groups for their
feedback, especially Caroline Bryant, Arpita Das, George Haynie, Ryan
Keverline, Michael Kricheldorf, Rose Thai, Evelyn Tipper, and Jacob
Wynne. We thank Monica Bruckner, Ashley Carlson, Kristin O’Connell, and
Cailin Huyck Orr at the Science Education Resource Center at Carleton
College for administrative assistance with module testing. All module
testing and assessment was conducted following approved Institutional
Review Board (IRB) protocols (Virginia Tech IRB 19-669 and Carleton
College IRB 19-20 065). This work was funded by NSF DEB-1926050,
DBI-1933016, and EF-2318861.