CXC’s Response to the Independent Review Team’s Recommendations

CXC’s Response to the Independent Review Team’s Recommendations

 

The Independent Review Team (IRT) commissioned by the Chairman of the Caribbean Examinations Council® (CXC®), Professor Sir Hilary Beckles, in response to concerns raised across the region with respect to the July/August 2020 CSEC® and CAPE® Examinations, suggested 23 recommendations to address these concerns, eight of which were for immediate action.

The following  section provides details regarding the recommendations made for immediate action and the initiatives implemented by CXC® in response to the recommendations.

The second recommendation made by the IRT was to reduce the cost for Requests for Reviews (RFR) for the recent 2020 sitting of examinations.

As such, the Council reduced the cost for Reviews by 50 per cent from US$30 to US$15. Additionally, those who had already paid the previous cost of $30 would be refunded. 

Based on monies received from the LRs for students who required reviews, those who CXC's Finance Department could clearly identify and who were required to be refunded for overpayment as well as for change grades were reimbursed. (Finance Mgr, HQ)

The IRT suggested that the Council revert to its pre-January 2018 position and practice, where a review included a remark.

All Reviews now include a remark. Reviews will continue to adhere to include a remarking of the script.  (SM, EAS)

A recommendation was also made to expedite Requests for Reviews, particularly for scripts that are borderline single grade changes.

A portal was created on 21 October 2020 as an additional and easy access point for candidates, which also facilitated the submission of immediate responses to candidates. Additional markers were also engaged to expedite the requests received.

The abandonment of grade diminution in CXC’s declaration “that a review of your script may result in your overall Grade either increasing, decreasing or remaining the same” was expressed under this recommendation.

Given the nature of the crisis, CXC® announced that a script reviewed would not result in a downward change of Grade.

The IRT recommended that the Council facilitate easier access and speedier response to Query Requests (QR), distinct from Requests for Reviews (RFR) and that a live chat and/or similarly more responsive medium should be used to allow stakeholders to make queries and receive timely responses.

Currently, Facebook Messenger and additional resources such as FreshDesk (Customer Support Portal), support@cxcsupport.freshdesk.com; cxcezo@cxc.org; and the support portal at devcxcedu.wpengine.com were employed to improve response times.

A total of 11,551 Requests for Reviews have been received for CAPE® and 10,055 Request for Reviews have been received for CSEC®. Additionally, 945 Queries were received for CAPE® while 2,174 queries were received for CSEC®.

EAS has reverted to only using the ORS for the submission of reviews and queries. This approach will provide oversight to the schools/centres and LRs. On the human resource side, additional temporary resources have been retained to support the EAS staff and teams have been established. Experienced persons, having worked on this aspect of the service, will lead respective teams. We are in process of engaging with the examining committees on the review and remarking to have the relevant markers identified and ready to participate. (SM, EAS)

The IRT recommended the execution of a reflection workshop – improved communication for Local Registrars about the grading exercise and function of profiles in grading.

CXC® made a presentation to Local Registrars on 21 October 2020 on “Understanding Your Grades”. Furthermore, training will be conducted for Local Registrars on Grading. Additionally, videos will be created for sharing with the public. One such video “Understanding Your Grades” was created and shared with the public on Friday 13 November 2020 (see below).

A workshop was conducted on 4 Nov 2021. This was coordinated through the DoO office. (SM, EAS)

A dedicated e-mail has been established and is in operation. The email is routed through Freshdesk. (SM, EAS)

EAS continues to engage with the LRs formally at monthly meetings, use of the dedicated email address and a shared WhatsApp group. Accessibility and responsiveness have improved. (SM, EAS)

The final recommendation for immediate action is for the execution of a damage control campaign across traditional, new and emerging media to inform a positive public relations perception.

CXC® has produced this interactive web portal with specific content and videos for stakeholders, based on their feedback. One such video “Understanding Your Grades” has been created and was shared with the public on Friday 13 November 2020. 

Virtual town hall meetings will be held to engage stakeholders, beginning with parents and students. The dates for the virtual town hall meetings will be published shortly. In addition, students' forums will be held for student council and association leaders. 

The Council values and appreciates all our stakeholders’ feedback and will continue to listen and engage with each stakeholder group as we work together.

The damage control document has been finalised.

The campaign is ongoing.

Formalize the provision and release of strategic training workshops for teachers to arrest the difference between teacher-awarded scores and CXC-moderated scores.

CXC’s Action:

Strategic training workshops to be conducted for teachers for improvement in School-Based Assessment.

Workshop sessions to include analysis of:

a) teacher and moderator score difference

b) share moderators’ comments across territories for each subject.

Q1-2021

  • SBA Training Commenced in January. They were 7 workshops conducted between January and March. They were conducted in the format of workshops and webinars. It is expected that there will be more webinars conducted by June 30.

 

Q2- 2021

 

The CPSM will continue to monitor the progress of these activities via Zoho and Spider Impact.

STATUS AS OF 2 NOVEMBER 2021

No additional training sessions were conducted as scheduled as there was “no schedule nor capacity to offer any further sessions in June 2021.” EAS “will work towards having further sessions potentially in Q1-2022 but this will depend on other cyclical activities.” (SM, EAS)

 

Institute initiatives and programmes to ensure a more rigorous and much sharper focus of teachers in preparing students for SBAs and provide requisite training on all aspects.

CXC’s Action:

Strategic training workshops to be conducted for teachers for improvement in School-Based Assessment.

Workshop sessions to include analysis of:

a) teacher and moderator score difference

b) share moderators’ comments across territories for each subject.
(To be completed in 1st quarter of 2021).

Formalise administrative training for teachers on preparing and uploading SBAs.

CXC’s Action:
Create video for self-paced training of teachers across region for the upload of SBAs. Videos will be useful for continued capacity building for new and existing teachers. 1st quarter of 2021

Engage in a series of education campaigns for students to help them anticipate, respond to and avoid consistent impediments faced in completing SBAs.

CXC’s Action:
See response to recommendations 1-3. 

As technological innovation for handling voluminous files is updated and refined, information should be shared with stakeholders including local registrars and teachers and requisite training conducted.

CXC’s Action:
Deployment of Data Portal (1st quarter of 2021).

Q1-Q4 2021

-   This activity has been integrated into the redesign of the corporate website with an aim of ensuring that territories are able to upload documents such as SBA’s without delay or error.

-   The e-SBA application was redesigned to provide scaling on peak load.  The application was deployed in Q2 and a video was also done which is available at https://www.cxc.org/esba/.  Further support was provided as needed. Centres reported improvement in file upload and reporting.  The LRs component of the system is currently being reviewed and upgrades should be completed by December. This is mainly backend infrastructure work that improves performance without major changes to functionality.

-   Customized data portals filtered by territory are scheduled to be available by Q4-2021.  This will allow the LR to connect and extract data related their respective territories without having to request from CXC.  Most reports provided in the Statistical Bulletin will be available once results are finalized.

Conduct an audit of the standardisation process for moderation of exams within and across countries.

CXC’s Action:
Immediate discussions with Local Registrars to commence process. To be completed in the 1st quarter of 2021.

Q1-2021- (Ongoing)

  • Process being reviewed & Standardization commenced

This activity is linked to #10 and #11 as the audit will include ongoing random selection of centres to ensure examination standards are maintained.

Formalise the provision of a series of workshops for examiners in each subject area on the grading approach and marking activities.

CXC’s Action:
Workshops for Examiners. Commence in 1st quarter 2021 and throughout 2021.

Commence Q1 through to Q4 – 2021.

Workshops for new and returning examiners are conducted in a form of orientation to maintain standard approaches to grading and marking.

Replenish and expand the exam item bank on a scheduled, regular basis for all subjects to avoid overuse and mitigate security breaches (Paper1).

CXC’s Action:

  • Item writing programme to commence (1st quarter of 2021)
  • Full Item Bank Replenishment Strategy to be implemented (2021-2023)

Q1-2 2021

  • The item bank manager is currently engaged in completing the backup of the existing items from Grademaker. The API that was supposed to be available to CXC to use did not materialize which has created an expected delay of 3-6 months. ITS is working with BLT to explore middleware that moves from GM-QTI into CXC-Surpass. To mitigate against further delays, CXC has advertised to bring on additional Item Writers to expand the electronic item bank. CPSM is actively monitoring activity closely as it is critical to the e-transition.

 

It is expected that the Item Bank Manager will be able to advance this requirement to meet the deadline of 2023 at which time there will be a fully replenished electronic item bank.

Engage in a strategic network campaign to strengthen and deepen the relationship between CXC® and teachers within and across territories.

CXC’s Action:
Stakeholder consultations, education and career fairs for students, Higher Education Institution conferences, Teacher led conferences, media engagements and speaking opportunities. (1st quarter and throughout 2021).

Q1 through to 2021

The communications manager and his team has continued to develop ways in which to improve stakeholder/teacher relationships across the territories. Some of the recommendations from the communications office was to engage in webinars and workshop of which was completed between January to March. Other engagements of a similar nature will be completed in June.

The Corporate Communications team supported teacher training workshops through design, marketing and digital promotions during Q1-Q3. These included support for:

 

1.  CAPE® Environmental Science Teacher Orientation Workshops (July and August)

2.  CAPE® Computer Science Workshop

3.  CAPE® Information Technology Workshop

4.  CAPE® Design and Technology Workshop

5.  Summer workshop Series for CAPE® Building and Mechanical Engineering Drawing, CSEC® Technical Drawing, CAPE® Agricultural Science, CAPE® Environmental Science, CAPE® Tourism, CSEC® Integrated Science, CSEC® Social Studies

6.  CVQ Webinar (April 2021)

7.  CSEC® Theatre Arts Paper 02 Workshop

8.  SBA Workshop Implementation Support Series (January - March 2021)

9.  ACCA Workshop Teacher Training Support Series (shared via Constant Contact)

 

Other activities included:

 

1.    Worked with Council of Community Colleges of Jamaica (CCCJ) to organise signing of MOU at launch of CXC’s 2021-2025 Strategic Plan

2.    Coordinated Registrar’s participation as Keynote speaker at The Mico University College Research Day, April 2021

3.    Coordinated Registrar’s participation as a keynote speaker at Caribbean Association of Principals of Secondary Schools (CAPSS) Conference in July 2021

4.    Coordinated Registrar’s participation in the University of Guyana and the Centre of Excellence in Teaching and Learning's (CoETaL) Second Biennial Conference in September 2021

5.    Focus group held with secondary school students and principals of Queen's College and Richard Ishmael Secondary School, Guyana, in October 2021

Develop a system with regular sampling at all centres.

CXC’s Action:
100 per cent moderation across all subjects, across all centres using an improved statistical mechanism for sampling. Completed and to be used at the next examination sitting. (3rd quarter of 2021).

Q3- 2021

This activity is schedule for Q3-2021 however, some discussion about how the sampling will be conducted for ‘on-site’ versus ‘off-site’ moderation during process review sections for EAS led by CPSM in April. The sampling system has been updated and being used in the current examination sitting.

Conduct Efficiency Audit of Local Registrars.

CXC’s Action:
Efficiency Audit to be conducted of LRs operations. To be completed by the 1st quarter of 2021.

Q1-Q2- 2021

  • An initial meeting has been scheduled on May 18 to review the LR’s manual with an aim of improving efficiency. This activity has been included in the balanced scorecard initiatives as part of the DOO schedule of initiatives for 2021. This will be done in collaboration with new Internal Auditor.

The Internal Auditor is currently completing the Internal Audit Charter, after approval the audit plan will be completed. The Efficiency Audit will form a part of this plan. An assessment will be done to ascertain the possibility of commencing the audits in 2021 or re-scheduling same to 2022. However, all LRs will be required to complete a pre-audit checklist by end of October 2021, in preparation for the full audit to be completed.

Create individual portals on CXC® webpage dedicated to significant class of stakeholders including students, teachers, registrars, testing centres and parents.

CXC’s Action:
Rollout of the redesigned CXC® corporate website which includes artificial intelligence and chat bots. This will include pages for each stakeholder. (Commence in 1st quarter of 2021).

Commence Q1 and would be completed Q3-2021

  • The redesigned website was launched in March 2021.The chat bots were not implemented in Q2 as planned. Due to COVID-19 the knowledge base the chatbots draw from is out of date as the information we are putting out there is changing quite rapidly, so the bots have to be “retrained”. A new ETA is Q1 -2022.

     

    In progress. The dedicated pages for portals were developed and shared with DOO for feedback which is awaited from EAS.

 

In addition, dedicated pages for portals are to be developed in Q2-2021 through Q3-2021.

Establish a research and development unit to undertake empirical research and rigorously mine the volume of data already being collected, to inform and strengthen CXC® in evidenced based and data-driven policy making.

CXC’s Action:
Research Unit to be established in the Western Zone Office (WZO). (1st quarter of 2021).

 

Progress continues to be made as relates to the establishment of the Research and Development unit. The following steps outline the activities that continues to be execute in order to facilitate.

 

·     Research and Educational Services Division Concept paper finalized in September 2021. The paper included the structure, objectives, outcomes and functions of the Research, Insights and Product Innovation department

·     Position of Lead Education Projects Specialist to be filled by November 2021

·     JDs for Manager and 2 Research Associate positions created

·     Five International Expert Researchers (USA, UK, South Pacific) for Research Advisory Committee contacted and short listed for Council approval (September 2021)

Undertake contingency planning as part of CXC’s strategic planning process.

CXC’s Action:
Develop and Implement Business Continuity and Disaster Risk Management Programme. (1st quarter of 2021).

Q1-2021 – Q2-2021 (Ongoing)

ITS/CPSM - There were 3 meetings with respect to BC/DR in which the Director (ITS) formulated a work plan. Further development has continued into Q2-2021 which now include an agreed schedule of activities to be completed;

  1. Capacity Building
  2. ERM-Framework; [Q-Pulse Risk, Supporting Audit + Risk, Reporting into the Governance Committee (AFC-Apr-2021)
  3. Risk Assessment;
  4. Risk Register Established; [Y1 - 2021]
  5. Process Evaluation
  6. Impact Assessment
  7. RPO, RTO establishment
  8. Mitigation Planning [Y2-2022]
  9. BC Plans Costed; Targets Agreed
  10. Simulations, Evaluations of Assumptions

9.      Monitoring Progress [Y3-2023]

All actions in progress.

ERM Framework was approved, first risk assessment has begun with the HR department, additionally work continues on the configuration and deployment of QPulse Risk, the software used to manage the risk process.

Strengthen biennial stakeholder satisfaction surveys to assess CXC’s compliance with its mandate.

CXC’s Action:

  • To enhance stakeholder satisfaction, stakeholder consultations, education and career fairs for students, Higher Education Institution conferences, Teacher led conferences, media engagements and speaking opportunities will be increased. (1st quarter of 2021)
  • Biennial stakeholder satisfaction survey as recommended.

Continuing discussions to ascertain a greater understanding of the changing needs of the stakeholder. In addition, the communication manager has requested CPSM to review previous surveys to reduce the length of the survey with an aim of having a more concise and succinct survey.

 

A meeting was schedule on June 09 to discuss the service charter but had to be reschedule due to the unavailable of critical team members from the communications unit.

The survey has been reviewed by the Pro-Registrar and the Corporate Communications Manager. A new survey instrument will be developed shortly.

 

Meeting held with Senior Manager to discuss KPIs and survey.