Program Evaluation

Overarching Finding #1: There is no clear “continuum” in programmatic structure or practices that are replicated across the TLC programs.

The TLC programs at the elementary, middle, and high school levels are largely siloed at the present time, and a common theme of disconnectedness and confusion regarding the program model was evident. While this evaluation focused on the elementary program, interviews with TLC staff at the middle and high school levels revealed substantive differences in the TLC model at the secondary level, which creates substantial confusion with regards to the overarching structure and vertical alignment of a cohesive district program continuum for students with Emotional Impairments. Importantly, the TLC program at the high school level is considered to be a general education “alternative” program; while approximately 50% of students in the high school program do receive special education services, the model is not designed as a special education program to meet the needs of students with emotional disabilities, contrary to the design and intent of the elementary and middle school models.

Overall, several areas for programmatic improvement were identified during this Program Evaluation, including the need for a clear vision for the program and fidelity in the implementation of evidence-based practices that connects to this vision. The District is also encouraged to develop clarity in the program description and entrance/exit criteria that is based on student level of need and type/intensity of specially designed instruction required in order to make effective progress. Additionally, staff interviews were also noteworthy for a palpable sense of confusion related to the roles/responsibilities of the teachers and various related service providers, most notably the School Adjustment Counselor (SAC) and Board Certified Behavior Analyst (BCBA). The specific time allocation and degree of involvement from these related service providers was insufficient to meet the needs of students in the TLC program. Furthermore, substantial challenges related to the lack of a collaborative, interdisciplinary team was underscored.

Overarching Finding #2: There is a significantly limited “therapeutic” component to the elementary TLC program.

Evidence of this limited focus on social-emotional and behavioral needs of students was apparent in the analysis of students’ IEPs, which showed notably minimal supports with regards to service deliveries. A recent decline in available staff resources has also largely contributed to the lack of a foundational “therapeutic” component of the TLC program. Historically, a full-time allocated School Adjustment Counselor (SAC) was an inherent part of the elementary TLC program. Due to the COVID-19 pandemic and significant strain on financial resources, the District made several budgetary cuts in the Spring of 2020, including the reduction of SACs across the District and the reallocation of remaining SACs (1 per building). All program staff and building administrators noted the detrimental impact that this had on the elementary TLC program. While all staff members expressed a strong desire to do more group lessons with an increased focus on social-emotional skills within the context of TLC, there were widespread concerns about the lack of allocated staffing and availability to provide an increased intensity in services. Likewise, the program has minimal/no access to a Board Certified Behavior Analyst (BCBA) to assist in developing both evidence-based class-wide behavioral supports as well as individualized Behavior Intervention Plans (BIPs) for all students in the program.

Overarching Finding #3: The lack of a robust multi-tiered systems of support (MTSS) framework impacts the District’s ability to effectively service students with emotional and behavioral disorders within the least restrictive environment.

Although the District has implemented many positive practices aimed at creating stronger tiered supports, continued efforts are needed to create a more inclusive culture and build upon the skill sets of all educators so that they can better support students with social-emotional and behavioral needs in the general education classroom. This is especially important in cases where current TLC students may not truly fit the program’s intended learning profile of students with Emotional Impairments. An analysis of student demographics showed some potential disproportionality, which may indicate systemic challenges in the District’s identification and placement process. Specifically, 80% of students in the elementary TLC program are identified as African American in comparison with a district-wide rate of 50.1%. This discrepancy mirrors that of national trends in data that show overrepresentation of Black students, and more specifically, Black boys, identified with emotional disabilities (U.S. Department of Education, 2016).

Importantly, the TLC program is functioning primarily as a substantially separate program with TLC teachers providing the vast majority of academic instruction, yet only 60% of students had identified academic IEP goal areas. Only 13% of students currently have services written on their IEPs to receive academic instruction within the general education setting. Staff interviews were also noteworthy for a shared desire to develop a more systematic process for determining how to increase students’ time in the general education setting. Continued Professional Development in the areas of PBIS and SEL are needed for all staff to ensure that students with emotional and behavioral needs can be effectively supported within the least restrictive environment.

Several detailed recommendations were generated based on the findings from this program evaluation that will be useful in providing the District with additional ways to further develop capacity and ensure continued high quality programming for students with substantial social, emotional and behavioral needs. Recommendations were organized into three main sections: Recommendations for District-Level Considerations, Recommendations for TLC Program Model and Structure, and Recommendations for Curriculum and Instruction. These recommendations underscore the importance of strengthening the district’s multi-tiered system of support (MTSS) framework in order to meet the academic, social, emotional, and behavioral needs of all students, with an emphasis on creating robust universal and targeted social, emotional and behavioral supports at Tiers 1 and 2. Specific recommendations for the TLC Program include developing clarity in the overall model of TLC with regards to student learning profiles and eligibility guidelines, as well as an emphasis on evidence-based Tier 3 therapeutic supports, program staff allocation and roles/responsibilities, and instructional models.

Overarching Finding #1: AIM Academy Program model alignment with Alternative Education (not Special Education)

The design of AIM Academy and the district-wide alignment to an Alternative Education program are significant factors of the program that would necessitate the District applying for categorization as an Alternative Education school. As such, the district lacks a special education pathway for students who transition from the Middle School TLC program to the high school level and may still require a more restrictive placement with strong academic, behavioral and therapeutic special education supports and who do not fit into the “At Risk” category as defined by DESE. The District will want to use caution when transitioning special education students from TLC at the Middle School to the High School to ensure their special education needs are the primary focus with the intent to service in the least restrictive setting. The District should avoid having the AIM Academy attempt to be both an Alternative Education program and a sub-separate special education program for students with Emotional Impairments as it can not conform to the two varying sets of state regulations. The district will rely on DESE criteria for programming that will guide the discrimination of what student profile should enter an Alternative Education program and seek alternative pathways for students requiring therapeutic special education services. The confusion lies in the difference between how the district should approach a disability versus a disadvantage.

Overarching Finding #2: Lack of Curriculum and Instruction in SEL, PBIS, Transition and Life Skills

The AIM Academy provides a strong core academic curriculum and credit recovery services for struggling learners in the areas of English Language Arts, Mathematics, History and Science. There are some initial signs of Transition skills being forged within the program, however; this is a benefit of a newly hired staff with a Transition Specialist background versus a long-term programmatic implementation plan. From a special education lens, the AIM Academy lacks comprehensive, systematic instruction and assessment in Social-Emotional Learning, Life Skills, Self-Determination, and Transition skills centered around each individual’s need (through an IEP). With consideration as an Alternative Education program, AIM will also want to develop these skills but from a different perspective (a student’s disadvantage). Along that same vein, the district may want to develop strong connections to community-based or work-based learning opportunities for At-Risk youth to create real life experiences they can draw upon.

Given the high number of special education students in AIM Academy, the absence of curriculum in the areas outlined above has an impact on the quality of students’IEP. There is a lack of comprehensiveness in testing and application for students identified as requiring a sub-separate, therapeutic program. The IEPs moving forward will require stronger representation of social-emotional learning, behavioral support, life skills as well as transition skills. The development of a strong life skills approach (money management, cooking, cleaning, laundry) is a necessary component of the AIM Academy curriculum for both profiles: special education students and students At-Risk. However, the distinction of why students lack the skills is imperative to “how” they are instructed and what type of support or program may be needed. Students who are not limited by their cognitive capacity or an emotional impairment but more so due to socio-economic disadvantages, cultural barriers and/or other societal disadvantages may benefit from the same curriculum but for reasons that many “At-Risk” students are facing, lack of access. Again, the confusion lies in the appropriate approach to the student profile, is it one of disadvantage or lack of opportunity or is it due to an identified disability.

Overarching Finding #3: Enhance Unique Learning Formats and Universal Design for Learning to improve Student Outcomes

The DESE Alternative Education Guide defines student learning time as learning that “can happen in a variety of settings and formats” such as: Directed Study,Independent Study, Service Learning and Work-based learning. The creativity offered through an Alternative Education program offers students a variety of options for work credit. The Team will want to coordinate this within the framework of Universal Design for Learning (UDL) to improve overall access for students. UDL targets multiple means of Engagement (why), Multiple Means of Representation (what), and Multiple Means of Action/Expression (how). As the district solidifies its alignment to an Alternative Education program, a heavier emphasis should be placed on ensuring a variety of learning formats, that consider UDL and how that can improve student engagement, are in place. Universal Design for Learning naturally presents opportunities for various learning styles and offers accommodations embedded into daily learning. For students with disabilities, UDL offers many of the prescribed accommodations found within IEPs that can support greater student engagement, improve understanding, and promote independence.

Several detailed recommendations were generated based on the findings from this program evaluation that will be useful in providing the District with additional ways to further develop capacity and ensure continued high quality programming for students with substantial social, emotional and behavioral needs. Recommendations were organized into three main sections: Recommendationsfor District-Level Considerations, Recommendations for TLC Program Model and Structure, and Recommendations for Curriculum and Instruction.

The district-level recommendations underscore the importance of identifying the DESE-aligned pathway that AIM will align with and therefore, vertical planning for the TLC elementary and middle school programs may support the structure and model of a different pathway for students who require a therapeutic special education setting at the high school level. Additional recommendations to improve the District’s Trauma-Sensitive and Culturally Sensitive practices were outlined.

Specific recommendations for the TLC Program Model and Structure include developing a clear vision for the TLC program across all grade levels/buildings.In the case of AIM Academy, the District would benefit from developing a clear vision for an additional program pathway at the High School for students transitioning out of the TLC middle school. As noted above, substantive differences in the TLC model were noted at the High School level, which creates substantial confusion with regards to the overarching structure and vertical alignment of a cohesive district program continuum for students with Emotional Impairments. Within AIM, there is a limited overall emphasis on the intended “therapeutic” nature of the program. As such, it is recommended that the District create further clarity in the explicit identification and cohesive implementation of evidence-based social-emotional and positive behavioral supports that are offered as part of the specially designed instruction of a special education program versus an Alternative Education program, although both may require therapeutic support. Of significant note, the absence of the BCBA is counter-intuitive to the design and intent of the program, which is described as supporting students with behavioral needs. A major strength of the AIM Academy is the ongoing presence of the Adjustment Counselor for frequent check-ins and to keep a pulse on students’ state of regulation.

A review of current students’ assessment records revealed that cognitive and academic achievement measures were consistently utilized to inform the eligibility process and development of the IEP. General measures of behavior were also prevalent in most students’ records; however, assessments generally lacked the use of specific measures designed for students with potential emotional disabilities. The District should consider adopting a consistent practice across all programs in the use of specific assessment tools geared towards determining the presence of Emotional Impairment. Furthermore, despite the behavioral nature of students referred to AIM Academy, there was no evidence of Functional Behavioral Assessments conducted, to instruct the development of BIPs, in the assessment records. The assessment records also lacked testing in the areas of Transition Skills such as Independent living and Employment. The addition of Transition Assessments will be pivotal to special education students referred to AIM (or a future TLC model).

Given the potential alignment with an Alternative Education program, the district will want to ensure that IEPs are comprehensively written for students in the AIM Academy and that resources are available to support students determined eligible for AIM Academy. As the District identifies a special education pathway for students out of TLC middle school it will be imperative that each student is appropriately placed and that the level of special education supports and therapeutic services are identified and practiced. It may benefit the District to provide a professional development series on IEP development that will subsequently shape the implementation of services for students at the high school level.

In an effort to increase the effectiveness and impact of paraprofessional support, the District would benefit from creating clarity with regards to expectations for paraprofessionals that support students in the AIM Academy.

All program staff, including teachers and paraprofessionals, should maintain current certification in a crisis management program (e.g., CPI, Safety Care) at all times. The profile of students in the AIM Academy may require a finely tuned set of skills in the art of de-escalation. The District may consider adding more training and monitoring of the effectiveness of strategies as it relates to time on learning as well as encourage staff to participate in ongoing honing of their personal de-escalation skills with a lens on cultural sensitivity.

From a special education standpoint, the District would benefit from supporting an interdisciplinary team model that encourages increased collaboration among various service providers (Special Educators, OT, SLP, BCBA, etc.) and focuses on embedding social-emotional and behavioral regulation skills across the entire day.

As mentioned in previous TLC program evaluations, the District would greatly benefit from creating a TLC Task Force to drive programmatic improvements in a systematic and cohesive manner across the entire TLC continuum from Kindergarten to 12th grade. Additionally, this task force can provide guidance and support to the AIM Academy as an Alternative Education program if needed.

Recommendations in the area of Curriculum and Instruction were considered first and foremost with the framework of Universal Design for Learning. The premise of all curriculum and instruction should fall under the umbrella of Universal Design for Learning with a focus on enhancing the three elements: Multiple Means of Engagement (why), Multiple Means of Representation (what), and Multiple Means of Action/Expression (how). Specifically, within the TLC program, the students' varying social-emotional needs, and academic needs impact traditional learning and teachers will want to focus on more high impact accommodations. Two areas were highlighted for the District to expand on: Social Emotional Curriculum and Instructional and Assistive Technology. Specific reference to models such as AIM: A Behavior Analytic Curriculum for Social-Emotional Development in Children, by Mark Dixon and Dana Paliliunas, which incorporates the research-based approaches of Mindfulness, Acceptance and Commitment Therapy, and Applied Behavior Analysis was cited. The district will want to attend to improved use of instructional and/or assistive technologies by students to support equitable access to learning based on Individualized Education Plans. Although Randolph is a one-to-one district with regard to personalized devices, safe use of devices can be a delicate balance of student need and attention versus students who may show patterns of unacceptable behavior. However, Assistive technology can be an extremely useful tool when considering student accommodations and modifications such as self-pacing of student work, ability to discreetly provide feedback, and for providing targeted personalized support (e.g., autocorrect, suggestive phrasing, research split screens, digital organizers, etc). The district is cautioned to intermittently ensure equitable access is promoted for students in AIM with regard to individual needs, accommodations, and/or modifications as well as to consider the necessary life skills of laptop use. Proficiency with online systems is a requirement in many areas and will be needed for students who may want post-secondary education or who may enter the workforce.

Overarching Finding #1: An Omission of“Therapeutic” supports and Social-Emotional Learning Focus

An essential piece of a “Therapeutic” program is the integrated use of therapeutic tools and behavioral strategies in conjunction with a proactive, systematic approach to Social-Emotional Learning (SEL). The TLC program at the middle school is inherently lacking these components. A School Adjustment Counselor (SAC) is assigned part time to the TLC program and the limitations of scheduling greatly impact availability to the TLC students. To that end, it is questionable if the level of access to a SAC is any greater than a student in an inclusion setting would receive. Furthermore, there is not a Social-Emotional curriculum utilized in the program nor is there a sense of a cohesive approach to address Social-Emotional Learning and the resulting behavioral impacts within the program. The SAC attempts to spend more time in the program but other school-wide responsibilities impact the presence in the TLC program. There is a necessity to interconnect the “therapeutic” supports and behavioral approaches (as it relates to finding #2) to create a strong foundation within the TLC program.

Overarching Finding #2: A Lack of Classroom Management Resulting in Reduced Time on Learning

The program at the Community middle school lacks a cohesive and systematic PBIS system that targets expected rules within and across all learning environments. Lack of program structure directly affects time-on learning and student outcomes and this was corroborated by direct observation and data collection during learning opportunities across the school day.In addition to a programmatic system, the “therapeutic” premise of the program would benefit from a stronger system (i.e., Check In-Check out system) that focuses on behavioral and/or social-emotional expectations at the onset of each activity. This approach serves as a priming procedure to set students up for success and on the latter end of an activity it also provides a self-reflection piece. The Middle School “TLC” Program Evaluation identified the need to develop stronger self-awareness, self-management, and self monitoring skills amongst the student population. Program alignment with the school-wide “ROCK” system would start to build a connection to the larger middle school community. A program-designed positive behavioral support system will develop a more consistent approach to classroom management and, in turn, improve student outcomes and time on learning. The TLC program is situated at the end of a hallway in the back corner. There are both positive and negative aspects to this design. One perspective would oppose the site of the program as it could create a feeling of exclusion and separation from the “general education” classrooms and peers. The other perspective, especially in light of the program's current status, promotes student privacy, dignity and respect by shielding their behaviors from the larger group of peers. Alongside the two academic classrooms is an additional room called the “Flex Room”. This room holds much potential from a proactive behavioral and therapeutic standpoint, however, it remains undefined and used in reaction to student behavior, which is in opposition to the Department of Elementary and Secondary Education’s September 2021 guidance on reducing and eliminating the use of time out space. Identifying how the “flex room” will be used and redesigning the space could greatly support more effective regulation through the development of personal coping strategies students learn to use proactively.

Overarching Finding #3: Staff Retention Concerns /Staff Training in SEL and Therapeutic approaches

Both acquisition and retention of highly qualified staff have a major impact on the overall success of the Community Middle School TLC program. Reportedly, the last two years have resulted in 100% and nearly 100% staff turnover rates.In the 2021-2022 school year there were no returning members of the TLC Team to the program. During the prior school year (2020-2021) it was reported that only one staff remained in the program for the following school year. Significant staff turnover in the program has resulted in an unstable and inconsistent program that lacks leadership and replicable systems. This is evidenced through the current staff belief that they have to “start from zero” to get the program operating successfully for students. The findings from the Lippitt-Knoster Model for Managing Complex Change shows vision, consensus and action plan as missing components of the current program design. These missing elements create confusion among staff and leave staff to unintentionally sabotage the intent of the model. Consideration of these large scale needs while systematically approaching an action plan will guide the District to overhaul the program. All staff reported that they feel supported through the district’s Professional Development opportunities and that they are provided sessions that are pertinent to the specific needs of a Therapeutic program.In alignment with that, the District will want to ensure that the carryover or generalization of the learned skills are transferred to the classroom setting. A program with high needs students requires a honed set of skills by the teachers and paraprofessionals. This creates a need for even more highly focused professional development, on-the-job training and feedback (i.e., Behavior Skills Training), and continued access to professional learning. Based on current observations, the professional development focus will need to be on developing a therapeutic setting with behavior systems and social-emotional approaches to learning.

Overarching Finding #4: Needed focus on Assessment, Transition Planning, & IEP Development

Through access to a sampling of IEPs from the TLC program and through discussion with staff, the assessment process for students in the TLC program appears very linear in its focus and lacks a differentiated approach to each student’s need. The School Psychologist solely conducts the psychological (cognitive) assessment and the educational evaluation for each student who requires testing for their three-year re evaluation. This seems to be a limited and pre-packaged approach to evaluating students. There was no evidence of transition planning or goals that align to the outcomes of a transition plan. Transition planning, according to the Department of Elementary and Secondary Education, states that “Secondary Transition is a time that begins when they turn 14 (or earlier, if the IEP team agrees)”. Furthermore, the Individuals with Disabilities Education Act (IDEA) states that secondary transition services are a "coordinated set of activities... within a results-oriented process, that is focused on improving the academic and functional achievement” of students with disabilities. Given the profile and needs of the student population in TLC, it would be imperative that the IEP Team begin Transition planning as early as possible. Additionally, it will be important to enhance the transition planning process with consultation, training, and/or support from a Transition Specialist who brings a different lens and focus on how to develop appropriate services targeting long term, functional outcomes. When working with students with Adverse Childhood Experiences (ACE) or Trauma, there is a strong therapeutic benefit to knowing your evaluator and, through staff reporting, the evaluator is mostly unfamiliar to the students only serving the role of test administrator. Consideration of a more sensitive approach to testing and moving toward including portfolio assessments or informal assessments to  supplement formal testing would help create a stronger picture of each students’ learning profile. The lack of Social-Emotional Learning measures within the program carry over to the IEP process that has limited or no focus on the student growth in the primary area that directs students to a Therapeutic setting, social-emotional and behavioral needs. Lastly, it was noteworthy that the shared IEPs were lacking goals in the area of SEL or behavior skill development.In essence, the two areas work in tandem and should be represented as such on student IEPs. Students’ maladaptive behaviors reduce when they are taught more socially appropriate ways (regulation) to meet their needs. The creation and implementation of an Individualized Education Plan for a student in a sub-separate Therapeutic Learning program should identify student-specific goals in the social and/or emotional realm and oftentimes behavioral goals that support their placement away from the general education setting where all students can access Tier 1 or Tier 2 supports.

Several detailed recommendations were generated based on the findings from this program evaluation that will be useful in providing the District with additional ways to further develop capacity and ensure continued high quality programming for students with substantial social, emotional and behavioral needs. Recommendations were organized into three main sections: Recommendations for District-Level Considerations, Recommendations for TLC Program Model and Structure, and Recommendations for Curriculum and Instruction. The district recommendations underscore the importance of strengthening the district’s multi-tiered system of support (MTSS) framework in order to meet the academic, social, emotional, and behavioral needs of all students, with an emphasis on creating robust universal and targeted social, emotional and behavioral supports at Tiers 1 and 2. Within this tiered support system, additional recommendations to improve the District’s Trauma-Sensitive and Culturally Sensitive practices were outlined.

Specific recommendations for the TLC Program Model and Structure include developing a clear vision for the TLC program across all grade levels/buildings as is also noted in the Elementary TLC evaluation. As noted within this evaluation, substantive differences in the TLC model were noted at the elementary and secondary level, which creates substantial confusion with regards to the overarching structure and vertical alignment of a cohesive district program continuum for students with Emotional Impairments. As previously mentioned, the middle school TLC program receives students from the Elementary TLC program, an intended model of a therapeutic classroom, and then graduates students into the high school AIM Academy whose philosophical approach is to serve the “at-risk” population through an Alternative Education approach. This marked difference intensifies the confusion for the expected model at the middle school level.

The collaboration between the SAC and the BCBA plays a pivotal role in developing the successful school-based therapeutic model. PBIS provides guidance on developing interconnected systems for behavioral support and mental health support to effectively coexist. The district is encouraged to refer to the Midwest PBIS Network for guidance on developing interconnected systems called The Advancing Education Effectiveness: Interconnecting School Mental Health and School-Wide PBIS. This is an implementation guide of the Interconnected Systems Framework (ISF) that will support the complementary nature of managing behavior while simultaneously providing therapeutic support.

A programmatic emphasis should be made on maximizing the availability of inclusion opportunities across the TLC continuum so that students with emotional and behavioral disorders have equitable access to instruction within the general education environment, as deemed appropriate by the Team. The district will want to ensure that, prior to entrance into the TLC sub-separate program, interventions at Tier 1, Tier 2, and Tier 3 are exhausted in the general education setting.

The District should also consider adopting a consistent practice across all programs in the use of specific assessment tools geared towards determining the presence of Emotional Impairment. Furthermore, conducting a Functional Behavior Assessment (FBA) should be part of the fabric of any program model that supports students with behavioral needs. The District should consider implementing a clear process for conducting FBAs for students who exhibit challenging behaviors with differentiated practices geared towards this specific student profile, including the implementation of trauma-informed FBAs, as well as identifying the roles of specific professionals that are typically involved in this process (e.g. BCBA, Psychologist, Adjustment Counselor, Special Education Teacher, etc.). A student’s right to an FBA or review of a Behavior Intervention plan is outlined in IDEA 2004.

All TLC teachers and related service providers would benefit from training and sample IEP models that reflect streamlined best practices for documenting services across the TLC continuum while staying steadfast to the guidance on Least Restrictive Environment (LRE).

In an effort to increase the effectiveness and impact of paraprofessional support, the District would benefit from creating clarity with regards to expectations for paraprofessionals that support students in the TLC program. Specific staff training on how to facilitate generalization of targeted social-emotional skills, how to effectively prompt students during whole-group instruction, how to implement positive behavioral supports, and how to support students’ academic needs without creating prompt-dependence or hindering instruction from the teacher would be beneficial to ensure that this role is utilized more effectively.

An integral component of a Therapeutic setting is the understanding of how to utilize de-escalation strategies as educators and how to assist students with developing a tool box of social-emotional, behavioral support strategies.

The TLC program at the middle school level would benefit from scheduling an interdisciplinary team model that encourages increased collaboration among various service providers and focuses on embedding social-emotional strategies, behavioral regulation skills, and consistent use of the Flex room across the entire day. directors.

Recommendations in the area of Curriculum and Instruction were considered first and foremost with the framework of Universal Design for Learning. The premise of all curriculum and instruction should fall under the umbrella of Universal Design for Learning with a focus on enhancing the three elements: Multiple Means of Engagement (why), Multiple Means of Representation (what), and Multiple Means of Action/Expression (how). Specifically, within the TLC program, the students' varying social-emotional needs, and academic needs impact traditional learning and teachers will want to focus on more high impact accommodations.Three areas were highlighted for the District to expand on: Social Emotional Curriculum, Math and Science Curriculum, and improved use of instructional and/or assistive technologies to support UDL and equitable access to learning.