top of page

Assistive App for Prevention and Treatment of Developmental Language Disorders 

A tool that helps to train future speech-language pathologists (SLPs) who work with any kind of communication disorder. This tool can help to earlier identify children's' communication disorder and offer treatment that leads to a better prognosis.

Healthcare
Entrepreneurial 
Assistive Technologies
DURATION
12 weeks
SCOPE
Product strategy | UX Research | UX design 
TOOLS

InVision | Figma | Lucidchart

Value Proposition

Deliver a user-friendly tool that offers Speech-Language students and pathologists an automated, smart tracking system for pre-linguistic behaviors at an early age.

Goals

​Clients' Goal:

  • Have a checklist based system to track children's pre-linguistic behaviors which improves the efficiency of linguistic clinicians.
     

  • Have a checklist based system which allows both linguistic clinicians and students to use it with different feature availabilities.

     

Designers' Goal:

  • Design an intuitive interface that takes low effort for clinicians to learn.
     

  • Combine recorded video or script with the checklist system to improve working efficiency. 
     

  • Make use of potential data. Propose data visualization method. 

Major Challenges

  • Lack of professional expertise in speech-language pathology (SLP)
     

  • Limited access to real users – the user of this product is really specific
     

  • Hard to search for any product reference, (as we didn't know at the beginning that this is similar to behavior analysis software)
     

  • ​Many people in the medical/health industry are not tech-savvy, and they are less likely to spend much time to learn a new tool. 
     

  • Privacy concerns about patients' information 

Achievements

  • We have fulfilled all the clients' requirements. 
     

  • We have exceeded clients' expectations with a more intuitive and efficient system.
      

  • We proposed new potentials for future research.

Feature Overview

Core Features

Login & Search for Patient

The app will be connected with the clinician's working account. Additional permission is required to guarantee the patients' privacy.

 

The clinician will be able to access patients' background information, session history, and any attached materials. The clinician can also add any new session progress (checklist) or new attachment.

Core Features

Checklist System

The checklist system allows clinicians to keep tracking on the children's prelingual behaviors based on the category of Content, Form, and Use.

 

In case if anything notable happens, the clinician will also be able to take notes in each section. 

 

For students who are working under the mentorship of a clinician, his/her form will be automatically marked as a practice to avoid any repeated data records.  

Core Features

List View, Video View & Script View

The checklist also allows the clinician to switch between checklists only mode, checklist with video mode, or checklist with scripts mode. 

​A new attachment could also be add from the checklist.

Advanced Features 

Goals

In the overview session, a clinician or student with special permission can set a goal to focus on special tasks. 

Any new goal will be shown on any new checklist until the task has been completed.

Advanced Features

Multiple Views 

In the overview session, a clinician can also choose to enable or disable automation feature and goal feature, this specific feature is designed to fulfill the different requirements for student's educational purposes and clinician's clinic use. 

Concept Exploration

Data Chart for Future Research

We also proposed a feature to provide data analysis potentials.

 

Previously, the offline system using Excel made it hard to collect and compare data, with this potential to show the children's learning curve, which could be interesting for future researches.

 

  • Behavioral data only – The use of data will NOT include any patient's private information.

Words From Our Clients

"....It looks incredibly intuitive, clean, and I really like the graphically presented reports you included."

“This looks SO cool!!! It really looks fantastic. Beyond the design and usability (which are great), I am so impressed with how much you understand the checklist/system. Are you sure you two don't want to be SLPs? :)”

About

Background information about designing for

pre-linguistic behavior tracking system.  

About

Contribution

Designers: 

Both designers participated and contributed to all research, design, test process and final presentation.

My Major Contribution:  

Research: Deep research to Speech-Language pathology, defined current user experience flow, defined content map, and journey map, initialized core-concept analyzation system. 

Design: Initialized information architecture, initialized wireframe, content iteration discussion and iteration, high fidelity mockup design, interactive prototype  

Test: Usability test manuscript, all feedback summarization.  

 

Product Management: Led all design with clients workshops, defined product scope. 

Manal Alyaba

Learning Experience Designer, MFA Candidate:

Brainstorming, product horizontal scanning, collaborated on the journey map, collaborated on the wireframe and high fidelity mockup design.  

Clients

Clients also worked closely with designers on providing expertise in communitive science and language pathology. 

Christina Reuterskiold, Ph.D., CCC-SLP

Project Mentor, NYU Associate Professor. 

Provide guidelines and insights on how linguistic pathology works with patients.  

 

Emily Matula, M.A 

Project Mentor, NYU Doctoral Candidate. 

Work with designers to provide resources for project iteration and usability research.

Anna Eva Hallin, Ph.D

Karolinska Institutet (Sweden) Postdoctoral Researcher, Original Mock-up Designer. 

Natalie M Kane, MS

NYU Master Candidate, Original Mock-up Designer. 

Process

WEEK 1-3

Research & Define

  • Value proposition

  • Current experience flow

  • Point & opportunities 

  • System analyzation

  • User stories

  • Content mapping with clients

  • Session observation

  • Subject landscape

WEEK 4-9

Design & Iterate

  • Brainstorming 

  • Storyboarding

  • Designing with clients

  • Information architecture

  • Wireframing (low-fidelity) 

  • Feedback session with clients

  • Content iteration (mid-fidelity)

  • High fidelity mockup

  • Interactive prototype

WEEK 10-12

Test & Plan

  • Usability research preparation

  • Qualitative usability testing

  • Feedback summarization

  • Product evaluation with clients

  • Future plan and the next step 

  • Finalize the project website

What is Pre-lingual Behavior Analysis? 

Children show pre-linguistic behaviors such as gaze at moving objects, copy adult’s behaviors as the signs to demonstrate their potential language abilities. A linguistic clinician will analyze such behaviors to identify the children's language development progress.   

Examples of pre-lingual behaviors: 

Note: Content, Form and Use are the three main categories of pre-lingual behaviors, a clinician needs to identify those behaviors and check them on the list in order to analysis a children's potential to learn a language.

Process

Details of how this project was developed through the process of research, ideate, iterate and the validation of the solution

Process

Phase 1:Research 

At this phase, our goal was to resolve the challenge:

Lack of professional expertise in the linguistic disorder ​

 

The client has provided us with a list of resources to look at that showcase the current work environment:

  • A developmental model for the analysis of child language samples

  • Current patient’s information/script

  • Excel sheet

  • Video of sessions

  • The original mock-up idea that the client is proposing

Current experience flow

Currently, the clinic is using an Excel sheet with a pre-existing checklist. Once a session with a child end, the student goes back to the session’s recorded video and watches it while marking and coding the observed pre-linguistic behaviors that manifested during the session. These Excel sheets are saved on the child’s file. Each new session a new Excel sheet must be created for each patient

Current Journey Map

The Checklist System

Below is a diagram that we have created from the current Excel sheet (the checklist). These are the behaviors that the clinician has to code from the session’s videos or script. There are no more behaviors that need to be added to the checklist by the clinician, but the clinician can add notes about the number of times he/she has witnessed a behavior or what missing opportunities during the session that led to not witness a specific behavior. The checklist is divided into three categories, content, form, and use. Under each category, there is the discretion of each pre-linguistic behavior.

**Reference content provided by clients on an excel sheet.

 Content, Form, Use.jpeg

Core Concept Analyzation

In a participatory design session with clients (who are also users), we have defined our main problems. 

The first step is finding the core problem that we are trying to solve. Which is to help speech pathologists to find an easier way to track children pre-linguistic behaviors. This tracking would be done via recording the session interactions, analyzing these records later and defining the advanced children’s behaviors. As a result of these actions, a child treatment plan should be put in place for future sessions, and track all the new pre-linguistic functions.

Secondly, it is important to understand why are we designing this tool, especially that there is a current tool is being used for a while to track patients’ progress (the Excel sheet). The problem with the current tool is that it is not efficient, not user-friendly, does not include any standardization and lacks a lot of advanced functions and navigation that assist in tracking previous records. For these reasons, we need to come up with a new tool that helps solve these issues. Additionally, we have to think about our primary users and the people who would be impacted by developing this new tool. 

 

Lastly, one of the important questions to think about is where, when and how this tool would be used. This tool would be used during the diagnosis sessions to record only by skilled pathologists, while under training pathologists would be mostly using it after the sessions to analyze the child’s behavior. So, in order to create a meaningful user experience, this tool should consider information automation, navigation, user interaction, and data collection.

Below is a diagram to explain all the initial requirements for this app:

**Collaboration with clients

Core Concept Analyzation based on What, Why, Who, Where/When and How

Content Mapping

In the same participatory design session with clients, we have asked them to define the most important features that they need in the new app.  We asked them to consider these questions:

  • What is the core information to evaluate a diagnose?

  • How do we measure success for this project/tool?

  • How can this tool influence who?

  • What are the most challenging things in using excel?

  • Anything else?

 

Below is a diagram to show the client’s responses. Please click the image for a clearer view. 

**Collaboration with clients

Content Mapping.jpeg

Users Stories

We've defined the user story based on the information we collected: 

Woman on Podium

Clinician’s supervisor:

"I want my clinicians to use this checklist so that they know in what level their patients are."

Clinician:

“I want to access the past records of my patient so that I know what pre-linguistic behavior they already have."

Young Doctor
Nurse Talking to Patient

Clinician:

“I want to code my patient’s behavior quickly, so I don’t waste my patient’s time."

Landscape Audit

In researching similar solutions we have chosen two paths. The first path was looking up how video coding works for behavior analysis and data. The second path was looking up software for patients’ medical records and how they work.

The majority of landscape audit was contributed by Manal Alyaba.

Video Encoders:

The Observer XT by Noldus – Observational Data Integration


It is a software for the video coding process with data analyzation. The software has various features that are not needed for the new pre-linguistic app. The software provides reporting and data exporting options in multiple ways. 

Why it does not work for our client?

  • The playback functions are not good for data analyzation 

  • Too many features and customization options might make it hard to use for the clinicians 

  • It is mainly used for research purposes, therefore it is not going to integrate with NYU’s clinic system 

Observer-XT.png
Behavioral-Observation-Research-Interact

Behavioral Observation Research Interactive Software (BORIS)

BORIS allows user-specific code video and audio files via uploading recorded videos or live observation sessions. It codes and stamps the time via one-click on the behavior or action. The good thing about this program it allows for customizable behaviors as a user can add unlimited numbers of behaviors.

Why it does not work for our client?

  • It is mainly used for research purposes, therefore it is not going to integrate with NYU’s clinic system  

  • Requires long learning curve, which makes it unsuitable for many clinics to accept it as a new tool.

  • It has so many features that do not match the pre-linguistic app requirements 

  • The analyzing visuals graphs lack clarity and readability

Datavyu

 

Datavyu is another video coding software that links behavioral researchers with their videos. It provides researchers to record their observations, obtain data for analysis.

Why it does not work for our client?

  • The big downside for this software is that users have to write down the observed behavior each time they are coding. So it is time-consuming for clinicians

  • It is mainly used for research purposes, therefore it is not going to integrate with NYU’s clinic system

Datavyu_edited.jpg

Medical Record System:

The main goals for looking into medical records software were to learn about how patients’ information is represented, how the patients’ information can be searched, and what information is required for medical records. So we have looked into the design of these three software, Advanced MD, Dr. Chrono, and Chiropractic

Data Visualization:

We have looked into a few websites to explore how different information can be presented on The Data Visualisation Catalogue.

Since data visualization is not part of our MVP, the reference we've found is only been used as a concept demonstration.

visualization options.png

Phase 2: Ideation & Iteration

During the ideation process, we focused on transferring our big ideas into an accurate and comprehensive application. 

 

Initial Proposed Content Map

Based on our communication with our clients, our initial understanding of the system and our research on the checklist system, medical record system, we upgraded our content map from the research into five steps experience content map.

 

Designing from scratch, this map served as the bone structure of our design. 

Thanks to our clients, who have been actively sharing insights with their expertise and provided timely feedback to our research, we categorized each feature based on the importance level of “most useful”, “must-have” and “nice to have”.

Flowchart.jpeg

Low-fidelity Sketch

We used InVision's Freehand Feature to create our initial low-fidelity sketch with the keyframes of our design.

 

Initial Client/User Feedback 

Being able to see a rough sketch of our product design worked well to communicate our ideas, based on this new perception to the system, our clients provided some insights which led to important changes to our design.

 

Here are the changes that were requested by the client:

 

Primary Experience
Our mentors realized that there might be a user experience conflict between professional clinicians and student clinicians. Although clinicians will consistently working with a patient in many therapy sessions, students are more likely to be working on the educational materials to practice. So, students will only be working with a new patient each time. Therefore, the system shouldn’t function the same for the students and professional clinicians. This new finding leads to a decision that shifted our primary user experiences. Instead of working on the checklist which records all the analyzation history, the user will start a new checklist every time they work on a new session. Even though the automation system seems to be a great idea to improve the efficiency of their working process, our mentor thinks it could be more beneficial for the students to analyze each behavior by themselves. 

 

Patient Record
According to our clients, there should not be a section to record the patient’s info. All patient information should have already been recorded to the system during registration or during booking an appointment. The only information that would be useful in this scenario are: 

  • Date of evaluation

  • Patient’s ID

  • Patient’s age

  • Number of siblings

  • Language exposure status at home and outside the home 

  • If the child is in a group situation

Patient Privacy
Our clients also were concerned about the video usage in terms of patients’ privacy, they are not sure whether we would have permission to store patients’ videos on a third-party app.  

Second Round of Ideation and Research 

Based on the feedback we collected, Manal and I had another round of brainstorming to ideate the design solution and research about the privacy concerns. In this round, we focused on: 

  • permission access to protect users’ privacy

  • adding a feature to initiate a new checklist

  • adding switches to allow students and clinician to use different features 

  • adding a script view for students who use a script to practice 

  • allowing people with special permission whether they were instructors or clinician to upload materials either a video or script

Proposed Experience Flow

We finalized our design into an experience flow for the team to clearly understand our design system. 

Experience Flow

Visual System

Based on our clients' requirements, we referenced NYU University Identity with a few modifications to meet specific feature requirements for the design solutions.

Style Guide.jpg
Solution

Solution

The final design solution, usability research, clients  review and next steps.

Usability Testing

Audience

  • SPL, Ph.D. students, Master students and Lab assistance 

  • Both are teaching students how to use the current checklist

  • One work with patients as well  

  • Meeting at the Department of Communicative Sciences and Disorders

Device

  • We have shared  the prototype with the users on a computer

  • Using Video and audio to record the interaction between the users and  the system followed by user interview  

** Privacy Permission:  video to word summary 

User test.png

We used Figma to develop our prototype. Figma allowed us to collaborate seamlessly online. We also connected the major pages and made an interactive prototype to test with users. 

Because we have limited access to potential users. In order to ensure our with research is well-organized and comprehensive, we also wrote a Usability Testing Manuscript to guide our process.

High-fidelity Prototype and User Feedbacks

We used Figma to develop our prototype. Figma allowed us to collaborate seamlessly online. We also connected the major pages and made an interactive prototype to test with users.

User Log In 

Scenarios:

1.  Entry Page

2. No Permission

3. Wrong Password

4. Logged Out

 

​User Feedback:

✓ Login with special permission that links to NYU account solved the privacy concerns 

✗  N/A

↝ N/A

Search for Patient 

 

Scenarios:

1.  Entry Page

2. No Permission

3. Wrong Password

4. Logged Out

 

​User Feedback:

✓ Simple and intuitive 

✗  N/A

↝ N/A

Add New Record

Scenarios:

1.  With Attached Files

2. With Saved Draft

3. Empty Stage

​User Feedback:

✓ Content is well understood without any explanation

✗  N/A

↝ The patient info could be different in a professional application (student vs. professionals)

New Checklist

Scenarios:

1. Default Checklist

2.Automation filter

3.Goal and Automation

4.Exit Without Save

 

Feedback:

 

✓ Layout is clear

✓Allows the user to count behaviors instead of just a checkmark

✗  A user was confused about what does the 3/5 means. But it makes sense once we explained (this is teachable) 

↝  Users hope to also be able to record missing opportunities on the list

Script View and Video View

Scenarios:

1.  Script View

2. Mark a Behavior

3. Behavior Marked

4. Video View

5 Submitted

User Feedback:

 

✓ Simple to use

✓ Able to mark the  timeline/script

✓ Link to checklist  

✗ Some student want to have all the list contents just like the checklist view

 

↝  N/A

Info & History

Scenarios:

1. Extend Patient Info

2.Session History

User Feedback:

 

✓ Like the extendable patient info.   

✗ N/A

↝ The patient info could be different in a professional application (student vs. professionals)

Progress Overview

Scenarios:

1.  Default Overview

2. Overview with Goals

3. Set a Goal

4. Confirm a Goal

 

User Feedback:

 

✓ Option to have goals  and smart view  

  The goal could be confusing into missing opportunities (need instruction)

↝  N/A

 


Data Visualization 

(Concept Exploration)

We also presented potential opportunities 

to use data visualization for future research 

(This is not final design)

User Feedback:

 

✓ Great Potentials 

  Don’t need an average comparison in the chart view 

↝  Want to see more options in a chart view – especially goal history

Next Steps

Designers 

  • Continue to refine our design solutions, propose additional features such as Count Missing Opportunities. 
     

  • Explore data visualization opportunities.

Clients 

  • Use the current prototype to apply for fundings.

bottom of page