Development and evaluation of interprofessional e-learning for speech pathologists, interpreters and translators

The delivery of safe, high quality and equitable speech pathology services to clients with limited English proficiency requires speech pathologists (SPs), interpreters and translators (ITs) to work together effectively. No studies have described the development and evaluation of interprofessional training delivered in an online format. This study outlined the development of targeted e-learning for SPs and ITs, and evaluated e-learning outcomes. Sixty-six SPs and 140 ITs participated in preand post-training online surveys. Knowledge, confidence and practice items were self-rated on 5-point Likert scales, while skills were assessed using scored short answer responses to a video vignette assessment task. Outcomes were compared statistically between time points, and identical items from SP and IT surveys were compared between professions. Relationships between participant demographics and training outcomes were also explored. After e-learning, both professions demonstrated statistically significant improvements in all measures of self-reported knowledge, confidence, past practice compared to intended future practice, and skills. Differences in the extent of improvement between professions are discussed. SPs and ITs with less experience in their current profession and less experience working with the other profession demonstrated greater improvement after e-learning. Interprofessional e-learning developed in consultation with SPs and ITs has the potential to be an effective, user-led and accessible mode of training delivery when establishing foundation-level interprofessional training platforms for SP and IT professions globally. More research is needed to validate these findings and explore the impact of improved workforce preparedness on the delivery of equitable, accessible and high-quality services to improve client outcomes.


Introduction
Of the world's population, three percent (244 million people) reside outside of their country of origin (United Nations Population Fund, 2015). Worldwide, speech pathologists (SPs, also referred to as speech-language pathologists, speech and language pathologists, speech and language therapists, and speech therapists) have a responsibility to ensure that they provide responsive services to clients with limited proficiency in the dominant language of their country, such as clients with limited English proficiency (LEP) in English-speaking countries. When working with LEP clients and families, SPs should collaborate with interpreters who convert spoken languages and translators who convert written languages (National Accreditation Authority for Translators and Interpreters, 2019). As language service providers sometimes qualify as both interpreters and translators, they will be labelled collectively as interpreters and/or translators (ITs) except when specifically referring to each role. ITs are essential members of the interprofessional team (Li, Gerwing, Krystallidou, Rowlands, Coz, & Pype, 2017;Sturman, Farley, Claudio, & Avila, 2018). SPs and ITs alike must develop preparedness to navigate the complex crosslinguistic and cross-cultural aspects of service delivery (Isaac & Hand, 1996). The SP-IT relationship is a key driver for safety, quality and accessibility in a range of workplace settings and practice contexts (Langdon & Quintanar-Sarellana, 2003).
Challenges have been documented regarding SP-IT interprofessional collaboration. Most studies exploring SP and IT perceptions have included SPs. Interpreters' perceptions have been less frequently documented (Huang, Siyambalapitiya, & Cornwell, 2019) and translators' perceptions have not been investigated exclusively. Difficulties relating to the ways in which SPs and ITs work together include SPs' self-perceived lack of confidence and competence to work with interpreters (Kostich & Weiss, 2007;Zhang & Crawford, 2018), their uncertainty and unrealistic expectations about the verbatim accuracy of interpreting (Clark, 1998), limited cross-cultural and cross-linguistic understanding (Kambanaros & van Steenbrugge, 2004), unclear role expectations (Huang et al., 2019), difficulties with real-time analysis of linguistic information (Roger & Code, 2011), insufficient time to complete all activities required in sessions with culturally and linguistically diverse (CALD) clients, and lack of briefing and debriefing (Clark, 1998;Zhang & Crawford, 2018).
The SP-IT relationship is arguably more difficult to navigate compared to the majority of other health professions (Huang et al., 2019) because of the SP's role in communication assessment and working with clients with reduced speech intelligibility. In communication assessments, interpreting requires a balance between the three-way interaction to render understanding of a client's communicative competence, and the more dyadic verbatim word-for-word interpretation to retain accuracy and integrity of the client's utterances (Roger & Code, 2011). Issues arise when ITs are requested by SPs to perform tasks that are outside of their scope of practice and Code of Ethics (Australian Institute of Interpreters and Translators, 2012). This may include assisting SPs with the analysis and assessment of communicative competence in an individual's language(s) other than English, or acting as cultural brokers to contextualise communication and feeding/swallowing assessment and treatment (Isaac & Hand, 1996).
It has been suggested that working with ITs requires dedicated training (Huang et al., 2019) on top of foundational SP clinical education (Kohnert, Kennedy, Glaze, Kan, & Carney, 2003). This is not routinely embedded into academic coursework (Santhanam, Gilbert, & Paveen, 2018) and varies between countries (Saenz & Langdon, 2019;Zhang & Crawford, 2018). SP students have been recruited in two studies evaluating training to work with ITs (Pechak, Summers, & Velasco, 2018;Quach & Tsai, 2017). Only one known study to date has evaluated training outcomes for qualified SPs in Australia regarding working effectively with ITs (Zhang, Crawford, Bernard, & Walker-Smith, 2019a, 2019b. It is also the only study that has investigated the outcomes of training for ITs to work effectively with SPs. SPs' and ITs' selfreported knowledge and confidence improved after face-to-face training and were maintained at two months follow-up. However, self-reported improved intent to implement optimal practices after training, like briefing and debriefing, decreased in actual practice two months later. Zhang et al. (2019aZhang et al. ( , 2019b suggested that the lack of implementation of optimal practices may have been due to organisational and systemic constraints outside of SPs' and ITs' control (Huang et al., 2019;Sturman et al., 2017;Williams, Oulton, Sell, & Wray, 2018) such as time restrictions, lack of ongoing practical/simulated learning opportunities, and power dynamics within interprofessional teams. The pilot study utilised face-to-face training and did not expand data collection beyond the paediatric healthcare setting. Thus, whether similar outcomes can be expected across different practice contexts and training formats remains unknown.
E-learning is education and training delivered in an electronic format, and is referred to synonymously as online learning, web-based training, internetbased learning and computer-assisted instruction (Rohwer, Motzae, Rehfuess, & Young, 2017). It is an increasingly used format in healthcare related training, and a recent systematic review concluded that it produces outcomes no different from face-to-face learning (Rohwer et al., 2017). The advantages of e-learning over face-to-face learning include but are not limited to: learner-directed time and place of learning, user-led learning, lower costs of training delivery, and being a consistent and standardised platform accessible to an international audience (Rohwer et al., 2017). Emerging e-learning research in the medical field has demonstrated that short e-learning modules involving video and critical reflection tasks can improve knowledge, self-efficacy and attitudes towards working with interpreters and LEP clients (Ikram, Essink-Bot, & Suurmond, 2015;Kalet, Mukherjee, Felix, Steinberg, Nachbar, Lee, Changrani, & Gany, 2005). However, these studies only recruited medical students rather than qualified medical professionals, and did not involve health professionals in other disciplines such as SPs. Outcomes of e-learning initiatives for ITs to work effectively with healthcare professionals has not yet been explored.
Considering the importance improving interprofessional collaboration between SPs and ITs, the present study aimed to develop targeted foundationlevel e-learning packages for both professions and evaluate the e-learning outcomes (knowledge and confidence to work interprofessionally, previous practices compared to intent to implement future practices, and skills to identify and problem-solve common issues in SP-IT-client interactions) for ITs and qualified SPs across a range of workplaces and locations. The authors hypothesised that SPs' and ITs' preparedness to collaborate interprofessionally would improve after e-learning with the same extent of practice and skill improvement between professions, and that training outcomes would not be affected by participant demographics.

Research design
A pre-post survey design was utilised. Low and negligible risk ethics approval and site-specific approval was obtained through the Children's Health Queensland Human Research Ethics Committee (HREC/18/QRCH/67) and Research Governance Committee (SSA/18/QRCH/85).

E-learning development
Two e-learning packages, approximately 120 minutes in duration for SPs and 90 minutes for ITs, contained webinar-style video presentation(s) with optional closed captioning. These were accompanied by electronic handouts and weblinks to additional resources. Content was presented through written, verbal, pictorial and video mediums. The contents of each package were as follows: IT package: E-learning packages were provided at no cost to participants. Training contents were adapted from the training materials used in a previous study involving face-to-face training (Zhang et al., 2019a(Zhang et al., , 2019b) and re-developed through an iterative process presented in Figure 1. The pre-determined choices of learning topics were derived from existing faceto-face training, interviews, consultations and previous training feedback (Zhang & Crawford, 2018;Zhang et al., 2019aZhang et al., , 2019b. Survey consultation results are detailed in Appendix A. The training development reference group provided further guidance to refine the e-learning packages. The group consisted of 14 different professionals in the following roles: a) SPs and other healthcare professionals with experience and interest working with ITs and CALD clients across metropolitan, regional, rural and remote locations in hospitals, community health, education, private practice, mental health, disability and Aboriginal and Torres Strait Islander services, b) SPs and ITs with experience in clinical education, workforce development, service delivery and service management, c) SP researchers with research expertise regarding working with CALD clients and ITs, d) spoken and signed language ITs, e) multicultural consultants, and f) organisational representatives from Speech Pathology Australia and the Australian Institute of Interpreters and Translators.

Participant recruitment
Across consultation and e-learning, SPs and ITs were invited to participate through the email lists, electronic newsletters and other e-communication methods like official social media groups of Queensland Health, Department of Education Queensland, Speech Pathology Australia Queensland Branch, Australian Institute of Interpreters and Translators (AUSIT), Australian Sign Language Interpreters' Association (ASLIA) and National Accreditation Authority for Translators and Interpreters (NAATI). Inclusion criteria required all participants to be providing services to clients located in Queensland currently or in the near future. Additionally, SPs were required to hold a tertiary speech pathology qualification, while ITs were required to be registered for AUSIT, ASLIA and/or or NAATI communications.

Data collection and survey development
Participants completed pre-and post-training online surveys containing a) participant demographics, b) self-reported measures within the parameters of knowledge, confidence and practice, c) short answer responses to a video vignette task assessing the parameter of skill, and d) two training feedback questions at post-training about self-perceived degree of learning and the usefulness of training. Self-rated parameters were measured using five-point Likert scales. Survey items within the parameters of practice, skill and training feedback were identical for both professions. Questions within the parameters of knowledge and confidence were designed to reflect the e-learning topics for each profession and were thus different for each profession. Knowledge, confidence, practice and skills survey items for each profession are outlined in Table 2 and 3. SP and IT survey tools were adapted from previous SP and IT face-to-face training survey tools (Zhang et al., 2019a(Zhang et al., , 2019b. Aside from changes to specific survey items to match the learning topics of each e-learning package, two parameter changes were made to the original tool. The parameter of attitudes was removed as it showed low reliability in psychometric evaluation of the original instrument, and remained largely constant after face-to-face training for SPs and ITs (Zhang et al., 2019a(Zhang et al., , 2019b. A reliability analysis for internal consistency was then performed using Cronbach's alpha coefficient on each of the self-rated parameters of knowledge, confidence and practice within the SP survey (20 items) and IT survey (13 items). Factor analysis was unable to be performed due to insufficient sample size. The item-total correlation was also calculated to determine how much each item within a parameter correlated with the overall score for the corresponding parameter. Cronbach's alpha showed each main parameter reached good reliability with alpha between 0.82 and 0.92. All item-total correlations were above 0.7.
The parameter of skills was added to the present study's survey tools as a systematic review identified that skills improve with e-learning (Rohwer et al., 2017), and objective measures of SP/IT training outcomes have not been used in the literature to date. To measure skills, SPs and ITs watched the same 18 second video of a short role-play SP-IT-client interaction during a communication assessment scenario, and were asked to a) list all of the problems they identify regarding both the SP's and IT's practice, and b) list all of the solutions the SP and IT can implement to prevent or solve these problems. Rather than using the video vignette approach as an e-learning tool only (Ikram et al., 2015;Kalet et al., 2005), data collected through this task were used for analyses. The first author developed a scoring matrix to score participants' responses in line with the shared content from both professions' e-learning packages. Independently of the first author, the second author scored a random sample of 25% of the IT participant responses with 88% agreement, and 25% of the SP participant responses with 92% agreement. Both authors discussed and resolved all disagreements, and final agreed scores were used in analysis.

Data analysis
Descriptive statistics for continuous variables are presented using mean and standard deviation or median and inter-quartile range (IQR) when data were not normally distributed. Normality was assessed using the Shapiro-Wilk test. Categorical variables are presented using frequencies and percentages.
Statistical analyses between pre-and post-training for knowledge, confidence, practice and skills were conducted using the Wilcoxon signed-rank test. Participants with missing values for one of the time points were excluded from the analysis of the respective variable. The change score for each of the items in the parameters of practice and skills (identical across SP and IT surveys) from pre-to post-training was then calculated. These were not normally distributed, thus change scores were compared between SP and IT groups using the Wilcoxon rank-sum test, presented using effect size r and 95% confidence interval.
The potential effect of various participant demographics on change scores pre-to post-training was explored using different statistical tests depending on the distribution. This included years of experience in the current profession (Kruskal-Wallis and post-hoc Dunn's test), experience working with the other profession (Spearman rank-order correlation after data categorised into 0-2, 3-5, 6-10 and >10 years as the distribution was sparse and skewed), as well as work location for the SP group (Mann-Whitney U test after regional and rural/remote were re-categorised into one category due to small proportion of participants selecting these categories). The effect of past training to work with the other profession on e-learning outcomes was explored but will not be presented as analysis was underpowered due to very small proportion of participants having completed training prior to the present study. The effect of workplace on SP e-learning outcomes was not assessed due to the high number of participants selecting multiple workplace combinations which limited the pooling of data into distinct workplace categories for analysis.
All p-values were two-tailed and p<0.05 was considered as significant. All statistical analyses were performed using the R statistical software (R Core Team, 2018).

E-Learning participant demographics
One-hundred-and-twenty-four SPs consented to participate in e-learning. Fiftyseven were excluded as they only completed the pre-training survey, and one was excluded as he/she did not complete any of the e-learning outcome parameter items at post-training. Of the 66 remaining SP participants, 97% were female and 3% male. Participants had between 0 and 40 years of SP experience (median 5.5, IQR 2.25-19.75). The most frequently selected level of experience working with ITs was 1-5 sessions. Eighty-nine percent had not completed any training about working with ITs prior to the current study.
Two-hundred-and-six ITs consented to participate in e-learning. Sixty-six were excluded as they only completed the pre-training survey. Of the 140 remaining IT participants, 79% were female and 21% male. Participants had between 0 and 37 years of interpreting experience (median 7, IQR 2-12) and 0 to 45 years of translation experience (median 6, IQR 2-10). The most frequently selected level of experience working with SPs was 0 sessions. Eighty-seven percent had not completed any training about working with SPs prior to the current study.
Please refer to Table 1 for additional e-learning participant demographics.

E-learning outcomes
Descriptive data and individual survey item descriptors in the parameters of knowledge, confidence, practice and skills at each time point are presented in Table 2 for SPs and Table 3 for ITs. Each item will be referred to in-text by its corresponding parameter-item code in the tables, e.g. K1 for Knowledge item 1.

Relationships between participant demographics and e-learning outcomes
When years of experience in the current profession was categorised into 0-2, 3-5, 6-10 and >10 years, participants with greater years of experience demonstrated smaller change scores. This affected K7, C3, C5, C7, P1 and P2 for SPs, and K1, K2, C1, C2, C3 and C4 for ITs specifically regarding years of experience as an interpreter. There was no significant difference in change scores observed between years of experience as a translator. For SPs there was a very weak to moderate negative relationship between experience working with the other profession and change scores for all items in the parameters of knowledge, confidence and skills (r between -0.09 and -0.42) except K4, as well as a moderate negative relationship for the parameter of practice (r between -0.44 and -0.59). For ITs, there was a weak to moderate negative relationship between experience working with the other profession and change scores in all parameters (r between -0.25 and -0.45).
For SPs, all but two survey items were unaffected by work location. K1 change scores were higher for those who worked in metropolitan settings, and P1 change scores were higher for regional, rural and remote settings.

Training feedback
At post-training, the median SP and IT ratings on a five-point Likert scale were both 4 'a considerable amount' (IQR 3-4) for the amount learned. Regarding the usefulness of training, SP and IT median ratings were again both 4 'very useful' (SP IQR 3-4, IT IQR 4-4).

Discussion
Following targeted e-learning, SPs and ITs demonstrated improved selfreported knowledge, confidence, practice (actual practice before training compared to intent to implement practices after training) and skills for interprofessional collaboration. ITs demonstrated greater improvement than SPs in their intent to implement briefing practices. SPs demonstrated greater improvements than ITs in their ability to problem-solve common issues, as well as their intention to ensure that briefing and debriefing are consistent and structured. Those with less experience working in their current profession and working with the other profession showed greater improvement after e-learning, though these relationships require further investigation through larger data sets.
The e-learning content developed in consultation with both professions in the present study aligned with the recommendations of a recent systematic review (Huang et al., 2019), whereby training for SPs focused more on the practical elements of how to collaborate with ITs, while training for ITs had more of a focus on understanding the SP profession, types of clients and clinical tasks. This was also reflected in the differing e-learning outcomes between professions: while ITs improved more in their intent to engage in briefing, SPs improved more in their intent to implement consistent and structured briefing and debriefing practices, and ability to problem-solve common issues in SP-ITclient interactions. While power dynamics in SP-IT interactions may require SPs to have greater command of these skills within sessions, future research should seek to validate the present study's findings and further investigate the effect of content variation on desirable and balanced practice and skills outcomes for both professions. It is also of interest that results of the consultation surveys revealed SPs' and ITs' most frequently selected topics of learning for themselves largely did not match what the other profession suggested for them. The contrast between learning needs perceived by self and others also emphasises the need for further dialogue between SP and IT professions about interprofessional learning, especially prior to developing training initiatives.
While not directly compared, SPs' and ITs' improvement in knowledge, confidence and intended future practices after e-learning mirrors that of faceto-face training (Zhang et al., 2019a(Zhang et al., , 2019b. It also supports previous findings of improved knowledge and self-efficacy (equivalent to confidence in the present study) for medical students after e-learning (Ikram et. al., 2015;Kalet et al., 2005). The effect of e-learning on skill development (Rohwer et al., 2017) was also evidenced in the present study. Furthermore, the known benefits of elearning as an accessible and user-led mode of learning (Rohwer et al., 2017) is reflected in participation rates. The present study's online mode of training was delivered to approximately 2.5 times more SPs and over 4.5 times more ITs than face-to-face training in Zhang et al.'s studies (2019a;2019b). This in part could be attributable to the practicality of completing training in the professional's own time rather than prioritising face-to-face training over competing clinical/professional tasks during work hours. The user-led aspect of e-learning also provided flexibility regarding which parts of each package were completed. Notwithstanding attrition rates expected of online tasks (46% SPs, 32% ITs), the majority of SPs (over 82%) and ITs (85%) who completed the package reported they had fully watched the key video components. Approximately 60% of SPs accessed some downloadable resources, and the same proportion of ITs accessed all downloadable resources. A reasonable proportion of both (60% SPs, 41% ITs) accessed web links to additional resources. Moreover, work location had minimal effect on e-learning outcomes as evidenced by pre-to post-training improvement differences in only two of 22 measures.
Aside from the potential costs of hosting web-based learning and managing potential technological difficulties, e-learning provides the added advantage of minimal ongoing costs compared to the human resourcing required to deliver ongoing face-to-face training. Despite the relatively small and moderate to low quality studies on e-learning in healthcare, blended learning delivered through multiple modes like e-learning combined with face-to-face learning, is suggested as the most effective method of training delivery (Rohwer et al., 2017). While face-to-face, e-learning and blended learning were not directly compared in this study, workforce development and service planners in the SP and IT professions need to weigh up the potential trade-offs between learning outcomes, accessibility and financial feasibility when selecting mode(s) of training for future initiatives. This is especially pertinent in countries like Australia where large geographic distances may prevent SPs and ITs from accessing face-to-face training. It should also be considered whether a tiered elearning approach beyond foundation-level content is required to further progress outcomes for those with greater experience, given the present study's findings that both professional and interprofessional experience may affect extent of improvement after e-learning.
With increasing diversity both within the workforce and the speech pathology client population, e-learning presents opportunities for international collaboration, cross-linguistic and cross-cultural learning beyond what can be achieved within specific workplace contexts. This aligns with aspirations regarding SP workforce preparedness for global citizenship (Millar, Carey, Fortune, Mathisen, Hill, Dukhno, & McKenzie, 2019). With qualified SP and student SP movement between countries for clinical education, temporary work assignments and permanent migration, the opportunity to develop consistent platforms of cross-cultural, cross-linguistic and interprofessional understanding between SP and IT professions will not only improve service responsiveness to meet clients' language and cultural needs, but also strengthen these professions' collaborative advocacy and action on language accessibility and equity.

Limitations
Due to time and resource constraints, maintenance of training outcomes was not evaluated, and no control groups were utilised due to foreseen potential difficulties with recruitment. Previous preliminary findings showed that predominantly knowledge-based training alone may not be sufficient for the implementation of optimal practices two months after training despite improved intent (Zhang et al., 2019a(Zhang et al., , 2019b. Future studies should aim to evaluate the effect of e-learning on maintenance of skill and implementation in actual practice, especially since e-learning can be accessed multiple times as a 'refresher' course or on an as-needs basis. Furthermore, due to the effect of small group sizes and skewed distributions, exploration of relationships between some demographic variables and training outcomes were statistically underpowered. Results could have been affected by potential systematic differences between included and excluded participants (those who did not complete e-learning). Despite statistically significant differences between groups in extent of improvement, effect sizes were small and require further validation to determine whether they are clinically significant. Outcomes should be investigated using larger data sets and more robust study designs.
The overall sample size for the e-learning portion of this study was larger than previous SP and IT training studies (Zhang et al., 2019a(Zhang et al., , 2019b and pretraining self-ratings spanned the full range of a five-point Likert scale. However, the low median ratings may indicate that SPs and ITs who volunteered their participation may represent the portion of their respective professions who have a self-perceived low level of preparedness to work interprofessionally. Likewise, consultation regarding e-learning topics for training development only captured the views of 69 SPs and 21 ITs providing services to clients in Queensland. Findings may therefore have limited generalisability to the rest of Australia and globally when considering differences in cultural and linguistic demographics, policies, service delivery and scope of practice. Future studies should consult SPs and ITs across multiple geographical locations regarding their learning needs, and compare the outcomes from mandatory and voluntary participation in training. Additionally, the number of ITs who participated in e-learning was more than double that of SPs. The limited availability of easily accessible and freely available professional development available to the IT industry for accruing NAATI professional development points, especially training that can be completed at any time in the context of the often ad-hoc nature of IT assignments, may have contributed to greater IT uptake. Thus, further consideration should be given to the facilitators and barriers to SP uptake of e-learning, such as package length and dedicated professional development time in routine work schedules.

Conclusion and future directions
This is the first study to outline the development and evaluation of interprofessional e-learning outcomes for SPs and ITs. Following targeted elearning, SPs and ITs demonstrated improved preparedness to work interprofessionally. ITs and SPs showed different extent of improvement in some aspects of self-reported practice and skills. In future, stronger study designs studies are needed to validate the present study's findings and investigate whether e-learning can produce sustained improvements in interprofessional collaboration over time. Further research should also explore whether training content should be tailored to SPs and ITs with differing levels of experience, and whether training impacts on client outcomes. SPs' and ITs' interprofessional learning needs across practice contexts and geographical locations can also be studied. With further research into its efficacy, e-learning may have the potential to become a consistent platform for both SP and IT professions globally upon which to build interprofessional capabilities in providing equitable and accessible services for CALD clients worldwide.

Declaration of Interest
The authors have no conflicts of interest to disclose. S2: What can the IT and SP do together to prevent or solve these problems?