By Derek G. Shendell, D.Env, MPH, Juhi Aggarwal, MPH, Allison Mulch and Jennifer Siehl.
In general, in K-12 and higher education enrichment initiatives, collecting evaluation data is critical to both a program’s future success and to continue to support student achievement. This includes topical training courses and academic competitions for individuals and teams. The NJSBA STEAM Tank Challenge is one such program.
STEAM stands for science, technology, engineering, arts (graphic and visual arts and design, including computer programs) and mathematics. Additionally, climate change/extreme weather event impacts and sustainability are emphasized in this competition. For more information, including the typical judging rubric, visit the STEAM Tank Challenge webpage found here.
It is particularly important to understand the successes, benefits, challenges and limitations of the modifications made to STEAM Tank to go from an in-person to a virtual synchronous format—for work among teams and for presentations by everyone at regionals and a subset at state finals—during the 2021 and 2022 competitions.
Therefore, the authors of this article representing NJ Safe Schools Program, in collaboration with staff of NJSBA STEAM Tank Challenge as well as NJ Audubon, conducted a survey from mid-January to mid-February 2023 before the 2023 competition regionals began early spring. The goal of the survey was to target teachers and administrators who served as coaches to K-12 teams in 2021 and/or 2022. The objective was to capture their experiences with NJSBA STEAM Tank during years 2021-2022, as presented as a virtual program during COVID-19.
In 2021, there were over 220 teams in regionals (first round), and in 2022 about 290 teams of two to five students, i.e., thousands of students. It must be noted that among the authors of this article, Shendell and Siehl served and continue to serve as NJSBA STEAM Tank judges, for regionals and state finals, providing critical feedback to uplift, support and help protect (via volunteer patent attorney to NJSBA) each team’s entrepreneurial ideas.
Survey methodology
A total of 110 participants were eligible for this online survey, which was emailed to their school district/school account. The survey was open Jan. 17 to Feb. 10, 2023. The authors downloaded data for review, management and analysis on Feb. 13. Participants were able to complete the survey on their own time using their own personal electronic devices in the environment of their choosing on the PsychData platform. To be included, participants had to access and start the online survey provided.
We reminded the teacher-coaches on three occasions by email to complete the survey. We told them the four-section survey should take no more than 10 to 15 minutes, and answers would be recorded anonymously and analyzed in aggregate; only stratification by self-reported gender identity was possible. Specifically, our invitation email stated, “Neither your school district/school nor personal names are recorded. No names of students are requested. This is consistent with current IRB approval of NJSS at Rutgers School of Public Health for educational activities.” In addition, there was no financial incentive offered for feedback.
The survey’s four sections focused on experiences in STEAM Tank, mental health as of mid-year/early winter 2023, how STEAM Tank improved the ability to teach seven skills, and demographics. Mental health was assessed using the Kessler Psychological Distress Scale (K6), which is used as a quick tool to assess the risk of serious mental illness or symptoms in the past 30 days (Yiengprugsawan, Kelly and Tawatsupa, 2014; Science of Behavior Change, 2022).
Data were analyzed through Microsoft Excel and SAS Analytics Software 9.4 (Cary, NC).
Survey results and brief discussion
A total of 35 of 110 participants (31.8%) partially completed the survey, and 25 of them (71.4%, or 22.7% overall) completed the survey. One person, however, did not answer a few parts of the third section. From the fourth and final section of the survey, on demographics, the participant sample was mostly non-Hispanic white (84%), female (80%), educated through undergraduate and master’s degrees (76%); only 8% each had only a bachelor’s degree, a doctoral degree or another post-secondary training credential reported.
Moreover, the participant sample represented the entire state: 44% from central region counties, and 28% each from northern and southern region counties. Finally, the participant sample ranged in age from early 30s to early 60s, with an average of 16.5 years (in year 17) of experience teaching in New Jersey (range 7-29) and an average of 18 years (range 10-29) overall teaching experience. In the future, more minority and early career teachers could get involved with NJSBA STEAM Tank Challenge across the state.
In the first section of the survey, participants clearly believed the Virtual NJSBA STEAM Tank Challenge helped continue the learning process during COVID-19. In 2020-21 (2021 Competition), 8 of 12 kindergarten to-fifth, 10 of 13 sixth to eighth, and five of eight ninth to 12th grade coaches said yes. In 2021-22 (2022 Competition), nine of 14 kindergarten to fifth, 12 of 16 sixth to eighth and nine of 12 ninth to 12th grade coaches said yes.
Please note these data add to more than 35 survey respondents each year because some coaches had elementary, middle and high school student teams (n=5) or elementary and middle school student teams (n=2). Please also note some teachers likely had increased responsibility in their school districts in 2021-22 (2022 Competition), given more student teams could meet not only virtually but also in-person during and/or after school as an approved activity.
In the survey’s second section, the majority of participants reported low levels of psychological stress as of the middle of the 2022-23 school year (end of first semester, Jan. 17-Feb. 10, 2023). Few teachers reported higher levels of stress via the Kessler-6 tool; however, many factors contribute to mental health. Overall, data support a need for more school-based mental health services for teachers, staff and students.
In the third section of the survey, most survey participants agreed the Virtual NJSBA STEAM Tank Challenge in 2021 and 2022 helped teach concepts of STEAM as well as climate change and sustainability. It is worth noting how slightly more participants “agreed” teaching concepts in technology and engineering were helped when compared to the other categories; and one and two teachers “disagreed” about sustainability and math, respectively. These observations likely relate to their students/grade level. [Please see the table below summarizing these data.]
In conclusion, based on responses of teacher-coaches of K-12 teams in virtual 2021 and 2022 regional competition (and some also the state finals round), the NJSBA STEAM Tank Challenge continues to be a successful educational enrichment experience for children throughout New Jersey in public K-12. Indeed, many of them also helped the teams of three to five students in prior years and continue to again in 2023 in person.
Dr. Derek Shendell is a Professor at the Rutgers School of Public Health and Director of the NJ Safe Schools Program, where Juhi Aggarwal earned her master’s degree in public health, majoring in epidemiology, and now works as a Research Teaching Specialist. Allison Mulch is the Director of Education for NJ Audubon. Jennifer Siehl manages NJSBA STEAM Tank Challenge competition for NJ School Boards Association (NJSBA); STEAM stands for science, technology, engineering, arts and math.
For more information visit the NJSBA STEAM Tank Challenge competition website.
References cited above:
Yiengprugsawan, V., Kelly, M., Tawatsupa, B. (2014). Kessler Psychological Distress Scale. In: Michalos, A.C. (eds) Encyclopedia of Quality of Life and Well-Being Research. Springer, Dordrecht.
Kessler-6 or psychological distress scale (K6+). 2001-2003. Science Of Behavior Change. (Accessed August 18, 2022).
