The DHS CX Team wants to ensure that you (the CX team hosting a session) can easily coordinate and conduct an observation session.
This web resource has all of the suggested steps for ensuring a successful observation session. We have broken it down into stages and activities. These stages contain descriptions, tips, and resources to guide you on your way.
Your typical leadership observation session stages
Stage 1: Coordinate
Coordinate with the Observer
You have an observer lined up for your research activity. This is your opportunity to share what your team is doing, provide any background information, and agree on a date and time. This might take some time, so its good idea to have a few different time slots.
We provided an email example below to help you get started with an outline.
Tips
- Keep it brief at this stage, you might provide additional context later with a briefing.
- The goals here are to give some insight into your teams work and dial in an observation date and time.
- Be clear on how long the session will take.
- Include 5-15 minutes for an introduction and a session ground rules breakdown.
- Include time for your team's post research debriefing activity.
Activities
- Provide background and details on your service
- Give context to research being conducted
- Share potential dates and times for the observation
- Request the observer select a couple of times
Downloadable Resources
Observer Coordination
Background: Leadership Observation Session of [Name of Agency and Program]
Research: [What research methodology are they observing]
Location: [Address or Video Conferencing Link]
*Optional personal note, example: For this session we can plan around your schedule. If you may, please give us your availability across the following dates (feel free to respond in-line here).
Please Select Preferred Observation Time and Date:
- Observation 1: [Date, time, and length of session]
- Observation 2:
- Observation 3:
Expectations
- We will be sharing the Observation Briefing to give you additional context
- We will provide the observation ground rules before the session begins
[Research methodology being observed (ex: Prototype development)]
Type of research you are conducting, the process (including schedule), and its goals
- [List of roles and a short description e.g., moderator (interviewer), facilitator (note taker), and research participant (refugee applicant)]
Questions?
CX Team Lead: [your email here]
CX Leadership Observation Initiative: cx@hq.dhs.gov
Stage 2: Brief
Brief the Observer
This is a chance for the observer to get excited about their UX research session and to learn more about what they we will be doing, their schedule, and the participation ground rules.
We created a briefing template for organizing this information in a standardized format. Share this via email and verbally if needed.
Tips
- Provide the briefing in person or verbally over a video call if possible.
- This is especially useful for observers who have not yet participated in an observation session.
- If you want someone to understand your research, it can help to provide extra information like a persona (a made-up person who represents your target customer) or a journey map (a diagram that shows all the steps a customer takes while using your service).
Activities
- Email the briefing to the observer
- Schedule a meeting for briefing (optional)
[Name of your DHS component or other context for observation]
Briefing Title: [What is the research or service being observed?]
Objective: [Goal of observation session. e.g. Observe a facilitated research session]
Methodology: [How is research/service conducted? e.g. prototype validation, service shadowing]
Date and Time: [Date and time of observation Session]
What is [methodology]?
[Short paragraph that describe the research or service observed and their goals]
Session Breakdown
Details
Observation Type: [Field Visit or Remote]
Location: [Physical Location or Tool (e.g. Teams/Zoom)]
Duration: [Total time it should take for the leader. Include Session & Debrief & Feedback]
Session Moderator: [Who will be leading the session. This will likely be one of the participants listed below.]
CX Host/Translator: [Person who will aid the leader and field questions during the observation session.]
Participants
[CX Host Team Member 1 e.g. Research Moderator]: [descr. of role e.g. Facilitator/host]
[CX Host Team Member 2 e.g. Second Researcher]: [descr. of role e.g. Notetaker]
[Participant of study/service]: [description of role e.g. Participant]
Observer: [description of observer e.g. CISA Executive Director]
Schedule
[Break down schedule here and describe steps]
[2-5min cover ground rules and expectations]
[2-5min introduction to Service Team]
[45min observation session]
[15min CX Hosting Team debrief]
[5-10min time permitting provide your feedback on observation process to Facilitation Team]
Total time: 1 Hour 20 min
Ground Rules and Expectations
Sessions like these are for you to learn about customers' experiences
Please don't use the session as an audit or judgement of CX Hosting Team's performance. For these observations to be an ongoing and fruitful collaboration, it is essential that staff feel comfortable with leadership's presence, and focus on the customer rather than you. This way, teams can run a session as they normally would and will not feel compelled to put on a show for you.
Some of the participants may be DHS staff as well. Most likely they will not be aware of your position as a leader during the observation. You must ensure their anonymity and protection should they say anything critical of leadership or the service. The design or service is the subject of testing, not the participant. You should use these sessions to listen for opportunities to revise processes, policies, or programs.
You are riding along, as a guest observer
We recommend that your formal title be withheld from the introductions. Instead, the moderator will describe you as a "guest observer" and may leave you un-named to avoid a chance of intimidating participants and influencing their responses.
When joining the session, it is okay to have your video on for the introduction (though it is not necessary) but after the introduction your video and microphone should remain off.
Please don't interact with participants during the session.
The team's moderator or facilitator is in charge of the session. You are present to listen, observe, and learn. The moderator may invite follow-up questions at the end of a session, but we recommend that you listen only.
Things to look out for when observing
Listen for questions that participants have about the tool they are interacting with or the process they are in. Note things that are frustrating or confusing for the participant, anything that is surprising to you, and questions you have about the method of the session, the process or tool they're interacting with, and what your big takeaways are from observing.
If you have questions and comments
If you have a CX translator assigned for the session and you need assistance or an explanation, please communicate with them discreetly so as not to disrupt the activity. If you are observing remotely, have a separate chat open with them and route your communication there.
Stay to debrief with the team
After the session's activity has concluded with the participant, teams typically hold a short debrief in which the team discusses notable events from the activity and their initial insights. This is a key part of the design process. Depending on the type of observation this is, it may be appropriate to participate in the discussion. Either way, please stay to observe this part of the process.
Be ready with your big takeaways
After the debrief, the facilitator of the session will request feedback from you regarding your takeaways from the observation session and process. This is your opportunity to provide your thoughts on what you gained from this session and your perspective as a participant in the observation initiative. We will use this information to gauge the initiative's effectiveness and to drive improvement in areas that need further refinement.
If your are unable to give feedback immediately after the session, then the Facilitation Team will reach out with a feedback prompt.
Stage 3: Observation
Run the Observation Session
Before the research session
Remind the observer their job is to watch, listen, and learn during the research session. They should not interact with the participants so they don't interfere with the research. Consider running through the ground rules if necessary and cover the details and purpose of the research activity before the session.
Run the Research
You made it to the observation! It’s time for you to run your research session.
Tips
- Keep the pre-briefing short and sweet.
- The observer should have their audio and video off after being introduced.
- Ensure the anonymity and protection of your participants.
- Remind the observer to stay for the activity debrief.
- Have the session host take notes of any issues that arise and what can be done to prevent them.
- Introduce the observer as a “guest observer.”
- Act natural, the process or design is being tested, not you.
Activities
- Lay out established ground rules
- Describe research method and goals
- Remind observer that they are welcome to stay for the debrief
Downloadable Resources
Stage 4: Debrief
Include Observer in Debriefing
Debriefing
Make sure to invite the observer to the session debriefing. This is a meeting where you talk about what happened during the session. Reiterate the purpose of the debriefing before starting. Welcome the observer to participate and provide their insights.
Send Thank You
Be sure to thank the observer for participating in your research session. Establish a good relationship with the observer and let them know about any future research that you are planning.
Tips
- Let the observer know the goals of the session debriefing.
- invite the observer to participate in debriefing.
- This is your chance to indirectly share any limiting process or technology issues.
- On the thank you, include ways to keep the conversation going. Do you have upcoming observation opportunities?
Activities
- Invite observer to stay for the session debriefing
- Invite observer to participate if you would like
- Consolidate notes
- Complete debriefing activity
- Afterwards, send thank you note
Dear [Observer Name]
Thank you for your participation in our Leadership Observation session of [program and research methodology]. Our team greatly valued your time and presence in the research study and your willingness to share your experiences for this event. Your comments were encouraging and extremely helpful.
*Optional Program/Project/Research Highlight. This is your opportunity to encourage the observer to participate again.
The [Project Name] project represents an important strategy with DHS’s overall Paperwork Reduction Act and Burden Reduction Initiative. The [Project Name] program provides personnel with a quick and clear process to prep for and adjudicate cases. This will facilitate a clear line of communication and expedite information sharing across the agency. In addition, our efforts to improve the customer experience for [Project Name] demonstrates DHS commitment to transform Federal Customer Experience and Service Delivery to rebuild public trust in Government.
We encourage you to participate in another observation session with us or reach out the DHS CX Team for other observation opportunities.
We are providing a list of our upcoming research schedules below. If none of the dates will work for you, we can coordinate around your schedule.
Thank you for your time and participation.
With warm regards,
[Service Team Host Name & POC Information]
