CBME in Entrada: PA Guide

Faculty have two different options for using Entrada, depending on their appointment status.

For those with a Clinical Faculty appointment:

Most clinical faculty are automatically provided with Entrada access. If they do not have access, please make sure they have a CWL account. Once obtained, provide the CWL username to your program assistant, who will submit an Entrada access request on your behalf.

For those who do not have a Clinical Faculty appointment:

Program Administrators can provide these faculty with an “external assessor” account in Entrada. (Please see instructions on the following page of this guide.) Alternatively, when a resident triggers an EPA they can add the assessor name and email address to immediately provide them with an “external assessor” account. With “external assessor” access the user can receive EPAs via email, but will not have login access to the system.

The email addresses used by Entrada are pulled from Workday, the central records maintained by Human Resources. If faculty need to update their preferred email address in Entrada, they must make edits to their “Work (Primary)” email address in Workday. Once updated, this information should automatically sync to Entrada within 24 hours. Department HR Administrators should also be able to assist with the update process.

Please note that if a faculty member has a UBC FASmail address already in place in Workday, they will not be able to override with an alternate institutional email through the self-service portal. To change a UBC email (aka FASmail) account to an alternate email (such as a Health Authority email), the faculty member will need to go through their Departmental HR representative. Please contact PGME Entrada support (entradacbme.support@ubc.ca) for resources and current information about the Workday email update process.

Faculty accounts are automatically provisioned for all staff with a UBC clinical faculty appointment, but for occasions when other health professionals need to complete an EPA, they can be added to the system as “external assessors.” These accounts do not provide access to the system, but do allow residents to send EPA requests via email. EPAs received from external accounts will appear in the dashboard just as with EPAs received from internal faculty accounts. To add a new “external assessor” account, navigate to Assessment & Evaluation in the Admin tab:

Then begin as if you were triggering a new assessment, but when you open the “Select Assessor” search field, navigate to the bottom and choose External Assessor.

You will then have the option to add a new “external assessor” account by simply entering first name, last name, email address, then clicking “Add Assessor.” You can then navigate away from the assessment in progress. The external assessor account will be saved in the system, but the assessment will not.

Academic Advisors (or CBD Coaches) can be setup if you wish to give specific faculty Entrada access for specific residents or groups of residents. Academic advisors will only be able to view the dashboards and assessments for their assigned residents.

1. On the home page of Entrada, navigate to Admin > Manage Courses.
2. Ensure you are in Administrator View, then, select your Program from the list. (Programs are listed as “courses” in Entrada.)
3. Under the tab Groups, click on the Green button Add New Groups.
4. Select a prefix for the group name and type it into the Group Name Prefix area. This prefix will be applied to all group names.
5. Select your group type. You can either define a number of empty groups and manually populate them with learners OR have the system automatically populate groups. To create empty groups, enter the Group Name Prefix, and a designated number of groups. Then click the blue Add button on the bottom right.
6. Click on a group name to populate it with residents and match to an Academic Advisor. Within the current setup, the term “tutor” refers to the academic advisor role. Adding tutors/academic advisors to a group allows those faculty to view the assessment data for any learners in the group.
7. To add residents to a group, scroll down to Select Members and use the dropdown menu to tick off members. You can add multiple learners at once by ticking off multiple checkboxes beside names.
8. After you have added the required names, click Proceed on the bottom right to save your work.

While Program Director access is provisioned by the PGME office as part of general onboarding, PAs are responsible for managing Competence Committee member access to resident dashboards in Entrada.

Once a new Competency Committee member is added, they have immediate access to all resident dashboards and EPAs for the program. They also have the ability to approve EPAs and promote a resident to the next stage of training.

To add or remove Competence Committee members:

1. 1. Navigate to Admin > Manage Courses then select the appropriate program.
   2. From the Setup tab, scroll down to the Course Contacts section and find the Competency Committee Members section.
   3. The search field will allow you to access any clinical faculty account in the system and immediately provision access to learner dashboards. Any new additions and existing members will appear in a list below the search box. Outgoing members can be deleted by clicking on the X next to their name.
   4. After you made the necessary updates, scroll down to the bottom of the page and click Save.

All clinical faculty are automatically provided with Entrada accounts as soon as their appointment is processed by HR. Shortly after their Workday account reflects their faculty status they will be able to initiate EPAs and receive EPA requests from residents. These Entrada accounts are generic faculty accounts and do not indicate program affiliation unless an administrator adds them to the “associated faculty” list for the program. Therefore, PGME recommends that PAs add faculty members to the “associated faculty” list in their Entrada contact lists.

While adding accounts to the “associated faculty” list does not impact a faculty member’s ability to complete EPAs, it does help residents discern which assessor name to select as they initiate a request. This is a particularly helpful feature in cases where faculty members in different specialties have the same name.

To add accounts to the associated faculty list, go to Admin > Manage Courses. In the Setup tab under Course Contacts, you will see a field which allows you to manage “Associated Faculty.” Any PGME Entrada account with faculty privileges will show up in this search field and can be added to or removed from the list. Once an update is made, scroll to the bottom of the page and save your changes.

Once a faculty member’s account is added to the associated faculty list, residents will see the program affiliation when they select an assessor. If a faculty account is added to more than one program both will appear in the search window.

Program Administrators and Program Directors have the ability to view pending and in progress assessments for all learners associated with a program. From this program overview page, you can also send reminders to complete outstanding assessments through the system. Open the Admin tab and select Assessment & Evaluation.

From the Outstanding tab, you can search for specific forms, or filter. The “Owner” is the person who is responsible for completing the form (usually the assessor, but it may be the resident if they have not submitted the form). Check the appropriate boxes and then send reminders as needed.

If a pending EPA needs to be directed to a different faculty member than the one initially selected, administrators can forward tasks on behalf of the resident.

Navigate to Admin > Assessment & Evaluation. Identify the EPA you need to forward, either by filtering or searching the Outstanding EPA queue. Open the EPA by clicking on the hyper-linked title.

Once you have opened an individual EPA, you will see a blue “Forward Task” button at the top, which will allow you to select a new assessor for the EPA. Once the update is complete, an email notification will be sent to both the old and the new assessor for that EPA form.

As of January 2023, PAs have the ability to forward outstanding EPA requests on to external assessor accounts. For more information on external assessors, please see instructions for adding external assessor accounts in this guide.

To delete an EPA that has not yet been submitted, navigate to outstanding tab under Admin > Assessment & Evaluation (as above for EPA reminders), and use filters to find the EPA you need to delete.

Tick the box under the trash icon and select Delete Task(s). The system will prompt you to enter a reason for deletion and this comment will display when the EPA is viewed in the Deleted tab. Confirm your action by clicking Remove Task.

To delete a completed EPA, switch over to the completed tab in the same dashboard. Once you find the EPA you need to delete, open the form by clicking on the form title. The full form view will give you the option to “Reopen Task.” Reopening the form moves it into the outstanding queue, where you can delete as described above.

Administrators can reopen completed EPA forms for a faculty member who needs to edit their submitted response. To do so, navigate to Admin > Assessment & Evaluation then switch to the Completed queue. Identify the EPA you need to open, either by filtering or searching the Completed EPA queue. Open the EPA by clicking on the hyper-linked EPA title.

Once the EPA form is reopened, it will appear in the Outstanding queue. Once the EPA is in Outstanding status, a reminder notification can be initiated and the assessor will receive a link to the form, allowing them to many any necessary edits before resubmitting. Please see the section above for more information on sending EPA reminder notificaitons.

Program directors and administrators can log meetings to maintain a record of conversations about learner progress. Additional documentation can then be associated with this meeting, allowing storage of other assessments such as ITERS, exam results, or other files within the system. Any files stored here will be visible to program directors, administrators, Competence Committee members and also to the resident. Confidential information for Competence Committee members should be housed in a separate file share location.

After opening a resident dashboard, click Log New Meeting button at the top right.

Select date, add comments if needed and create the meeting. Many programs choose to create a meeting for each Competence Committee and add all associated files together. Once the meeting log is created, use the action tabs to upload files, edit or delete meeting log.

Residents have access to their meeting logs and accompanying documents from the CBME Dashboard “Supporting Documentation” button.

UPDATED in April 2022: Residents can now upload their own documents to the Meeting Log/Supporting Documentation feature.

Occasionally, the program will need to complete and submit a form on behalf of a resident or faculty member (if, for example, the form was completed on paper). PAs and PDs have access to do this on behalf of a resident.

• Navigate to Admin > Assessment & Evaluation.
• Select the Record Assessment button.

• Select the resident, assessor, and date of encounter.
• Complete the form on behalf of resident or assessor and submit.

Program directors, administrators and Competence Committee members can access a learner’s CBME Dashboard for aggregate summaries of EPA completion data, as well as a learner’s Assessments page, which provides a list of individual EPA forms completed on the learner. For a more comprehensive guide of the CBME Dashboards, including the Visual Dashboard (new in 2022), please see the Competence Committee Entrada guide.

1. • To access the CBME Dashboard and Assessment pages, click on the Assessment & Evaluation icon located in the top right corner.

   • Select the My Learners tab. All trainees currently enrolled in the program will be found here. If your program needs to update this cohort list, please contact entradacbme.support@ubc.ca

You can view either a resident’s CBME Dashboard or Assessments. For an explanation of the Visual Summary dashboard, please see the following section.

In April 2022, an additional visual summary option was added to the CBME dashboards. Program directors, administrators and Competence Committee members can all access the full visual summary dashboard, and residents have a visual summary of their own EPA data.

• After logging into Entrada, click on the Assessments & Evaluations task list icon at the top right of the page.
• Select the My Learners tab to access the main program dashboard.
• From the tab menu below the filter options, select Visual Summary.
• You will be directed to the Visual Summary dashboard.

Normative Assessment View

The Normative Assessment view displays a summary of resident progress for the entire program. Sort by total completed EPAs, achievement rate based on entrustment rating (i.e. o-score), progress percentage, and achieved vs. total completed EPAs. Use the drop-down menu to view a summary of this data across all stages of training or by specific stage.

Resident Metrics

To view assessment data for an individual resident, select the “Resident Metrics” tab on the left of the visual summary. Select a resident name from the drop-down menu, which lists all trainees sorted by stage of training. (Residents also have this version of the Visual Summary on their own accounts.)

The blue, green and red boxes show the amalgamated metrics for EPAs completed by the selected resident. The chart to the right visualizes the weekly EPA acquisition rate over the last six months.

A date filter option can be accessed via the calendar icon. Click this to highlight assessments in a certain time period. The assessments in the selected time period turn from regular round points to diamond shaped points.

Clicking the book icon opens a table displaying detailed information for EPA observations. Sort or search this table by selecting column headers or typing into the field below. This information can also be seen by hovering over individual EPAs in the o-score chart above.

Note: PDs, PAs and Competency Committee Members all have the ability to access resident profiles and to promote learners by EPA or by stage of training. Deciding who is responsible for marking EPAs as approved should be determined at the program level.

1. In the residents’ CBME dashboard, select the grey circles to mark an EPA as achieved and approved by the Competence Committee. A pop-up dialogue will appear to confirm the EPA as completed.
2. To approve a learner’s entire stage of training, follow a similar process but click the grey circle beside the stage title, above the list of EPAs.

View of approved EPA on the resident dashboard:

View of approved EPA on the program dashboard:

Update:

As of May 31, 2024 the “Triggered/Completed Assessments by Faculty” report is available and functioning for CBME data completed for Royal College programs in PGME. The “Timeliness of Completion” report is now also reconfigured to support EPA completion data.

If you have any questions about your program’s faculty data, please be in touch with PGME Entrada support at entradacbme.support@ubc.ca

To extract faculty EPA completion per program, go to the Admin tab, choose Assessment and Evaluation. In the Reports tab, scroll down to the Timeliness of Completion or the Triggered/Completed Assessments by Faculty Report options listed under Assessments.

Please note that none of the other reports listed in the system are active, functioning, and vetted for the current implementation of the CBME module at UBC.

In addition to the CBME dashboard features outlined above, program EPA completion data is available in an extractable CSV file report called the “Assessment Tool Data Extract.”

Please note that none of the other reports listed in the Assessment & Evaluation dashboard function for the CBME module and should be disregarded.

From the Admin tab, choose Assessment and Evaluation.

In the Reports tab, scroll down to the Assessment Tool Data Extract option under Assessments. Then fill out the report form with your program, curriculum year, form type and EPA. Each EPA will produce a separate CSV file, which can then be collated and adapted for program use. Note that Context Variable and Milestone information will vary on each form and spreadsheet.

“Prompted responses” are any flagged items on an EPA. At UBC, EPAs include a professionalism or safety flag, but faculty members can also leave a flagged comment with feedback on the EPA form. When a resident’s EPA is flagged, an email notification goes out to the PA and PD, but all current program prompted responses can be viewed and managed within Entrada.

To access them, click on your Assessment & Evaluation badge, you will only see a Prompted Responses tab if you have any prompted responses for your program.

Once you have clicked the Prompted Responses tab, you will see a table displaying a list of your prompted response notifications, the course (i.e. program) it belongs to, and the owner, target, and review the status of each task.

To see details about any of the flags, click on the title of the EPA and scroll to the bottom of the form. In the Prompted Responses Section you can leave comments and check to see whether another program staff member has reviewed or commented on the flag.

Comments can only be viewed by other prompted response recipients and administrators associated with the program. These review indicators and comments are not visible to residents or assessors.