¹College of Computer and Information Sciences, Qassim University, Qassim

²College of Computer and Information Sciences, Riyadh Saudi Arabia

Corresponding author Email: ialturaiki@ksu.edu.sa

Article Publishing History

Introduction

Patient experience is deemed an integral component when measuring health outcomes or evaluating the quality of the care provided.  Current research suggests patients who have a good healthcare experience usually form more positive attitudes towards the healthcare system, which increases patient compliance, enhances care continuity, and improves overall healthcare outcomes Doyle et al. (2013). The first step in improving a healthcare system is to determine what a patient need are and how those needs are best met; this is the foundation of patient centred healthcare M.D & Lundeen (2015). Measuring patient experience is a crucial component in the goal of assessing the quality of healthcare. It allows health organizations to determine the shortcomings in service, therefore, improve the overall care paradigm pursuant to meeting patients needed Jenkinson et al. (2002). Yet, understanding and evaluating patient experiences can be challenging and they may not be easily incorporated into reality, as the answers are often subjective and specific.  Traditional methods used to gather the views of patients and service-users such as surveys, focus groups, interviews, and complaints Doyle et al. (2013) have been criticised for having a variety of deficiencies and limitations: survey length, infrequent sampling frequency, slow feedback, and a failure to integrate results into improving care Robert & Cornwell (2013).

Another problem with these methods is that none of these options measure the patient experience in real-time. The more time that passes between a service being rendered and a patient being asked to opine about the service, weakens the accuracy of the findings derived from questionnaires and other forms of information-gathering Stull et al. (2009), Bjertnaes et al. (2012).  The patients may forget how they felt at the time that the service was being provided, or subsequent events may have coloured their memory of it.

This work therefore proposes to develop an approach to measure patient experience while they are in the process of receiving healthcare services, i.e.  in real time.  The task involves developing a web-based platform and a mobile application.  The platform gives the patient experience admin the ability to map the patients journey through healthcare and identify points of pain or frustration in the process.  The mobile application senses the location of the patient by using iBeacon technology, then presents the relevant questions based on their location.  The general concept is that when the patient enters the clinic and comes close to the proximity of the pre-installed beacon, the application will be able to ask some questions to the patient about a service provided in that region.  The goal is to help stakeholders to identify which aspects of care are in most need of improvement.

Presenting a solidified definition of the term patient experience is complex, as it has until now been defined ambiguously and diversely by the various institutions and research studies that have attempted to understand it Wolf et al. (2014).  Thus, it has no formal definition.  In addition, there are multiple terms employed in health care, and thus only experts can know the distinctions between them (e.g., satisfaction, engagement, perceptions, and preferences), which makes conceptualizing this term difficult.  The healthcare environment is simultaneously undergoing rapid changes, which causes definitions to change over time Wolf et al. (2014).  Finally, it is relatively new as an area of focus M.D & Lundeen (2015).  The Beryl Institute, which is a global leader on improving the patient experience in healthcare, defines patient experience as the sum of all interactions, shaped by an organizations culture, that in- fluence patient perceptions across the continuum pf care Institute (2019). This definition has since been incorporated (with or without adaptations) into a number of healthcare facilities globally as their own definition of patient experience Wolf et al. (2014).  Cleveland Clinic, which is a pioneer in the patient experience, defines the patient experience as, first, providing safe care; second, delivering high-quality care; third, in an environment of exceptional patient satisfaction; and, finally, in a value-conscious environment M.D & Lundeen (2015).

The patient journey originates from a set of interactions between a patient and hospital staff, irrespective of whether they are medical or administrative staff. This journey is strictly personal and implies the patient’s involvement on different levels (rational, emotional, sensory, physical, and spiritual) Trebble et al. (2010), Trebble & Hydes (2011). The journey is worked out based on mapping a consecutive series of touch points between the patient and the service where the patient experience has been actively shaped Bate & Robert (2006), Bessant & Ma- her (2009). It is a multi-stage journey with many different channels and touch points along the way Druckenmiller (2016). In the context of healthcare, the touch points of the patient journey are classified into three stages:  the before, during, and after 1.  In the before stage, the assessment is based on the ease of making contact with a clinic, booking an appointment, etc.  While in the during stage, it is based on an appraisal of their health condition, care, treatment, etc.  Lastly in the later stage, it is based on the process of arranging their next appointment and appointment reminders, etc.

Beacons can be described as devices which transmit radio signals at regular, defined intervals. Bluetooth beacons characteristically use Bluetooth Low Energy (BLE), a short-range radio technology which is both effective and useful because it uses extremely low levels of power. Any smartphone or tablet which is in range of the Bluetooth beacon’s transmission area can pick up BLE signals. Powering a beacon can be managed using two processes: (1) using of a fixed power source and (2) using batteries with a typical lifespan of between six months and two years He et al. (2015). When the iBeacon is activated, it sends a data packet along with the signal it is transmitting.  This data packet is frequently called an advertisement packet and is regularly transmitted by the iBeacon, providing location and identity information, namely, UUID (Universally Unique Identifier), Major and Minor values.  These values can then be used to place a number of beacons in the most appropriate positions for the layout of a shop or a supermarket.  The manufacturers of the beacons give the values themselves, and they can be amended relatively easily, by using the Software Development Kits (SDK) of the manufacturer. In this work, we used a Bluetooth Low Energy-Based Beacon (BLE) to measurement of patient experience.  We choose Beacon technology since it overcomes certain drawbacks associated with other technologies such as: radio-frequency identification (RFID) and near- field communication (NFC). For example, it does not require any extra device, such as a scanner for scanning cards barcode systems.  In RFID the card must to be brought near to the reader for it to be scanned manually; however, the use of beacons system does not require any manual action.  In NFC the tag should be brought to within the range of 5cm whereas a BLE Beacon range is up to 100cm.

Patient experience is a primary quality outcome for health services, which may be utilised to enhance quality, governance, public accountability and patient choice Ahmed et al. (2014). In this section, we highlight certain research pertinent to the measurement and enhancement of patient experience.In 2014, Benson et al.  Benson & Potts (2014) adopted a novel approach for measuring patient experience, called howRwe.  This is a questionnaire-based method for rapidly and effectively collecting data concerning patients assessment of their professional medical care. howRwe is designed to diminish the necessary survey reaction time, while tightly empha- sising criteria that may be adopted to enhance the patient experience.  It comprises of just 29 words and easy readability, therefore it may be quickly read and comprehended by almost all readers.  Wheeler et al (2015) described how, in Ontario Canada, cancer patients care during the early treatment stages has been improved through the adoption of Diagnostic Assessment Programmes (DAPs).  DAP’s fundamental concept is patient navagation.  The objective of navigation is to assist and support patients who have recently received a cancer diagnosis, guiding them through medical and administrative processes that are occurring during a time when they and their families are potentially vulnerable and confused.   In DAPs, every patient is provided with an individual professional contact, known as their navigator, whose function is to guide and inform the patient during their transition from diagnosis into treatment and care, thus facilitating this process.

Yang et al. Yang et al. (2015) proposed adopting an iBeacon-based indoor positioning system in hospitals, assisting patients with discovering their departments or wards, thus enhancing their treatment experience.  This was designed according to the three-layer architecture of the Internet of Things: a network layer, facilitating data transmission through- out the location; a perceptual layer, interpreting individual user requirements, as well as an application layer, which delivers and displays this information on the receiving device.  A final aspect is that Floyd Floyd (1962), the shortest distance algorithm, is used to identify the patient’s nearest department or ward. Presented as a result of the experiment, hospital indoor positioning can be realised through the system, with its application saving both time for patients and required manpower, as well as conserving hospitals material resources.

Additionally, Lin et al.  Lin et al. (2015) implemented this technology (iBeacon) in National Taiwan University’s emergency room. In this case, the system was adopted for continually monitoring patients location in the facility.  Additionally, the system displayed the patients clinical information and information about the staff attending them.  Received Signal Strength was used to estimate the patients locations. This uses a signal generated by mobile devices to allow systems to determine their distance from the receiver, which may be used to deliver push notifications, in this case location and navigation information.  The systems accuracy was determined by applying a 95% Confidence Level, resulting in a range between 95.9% and 98.55%.  This provided an overall net accuracy of 97.22%, which is satisfactory for indoor location determination.  The tested system comprised of four elements: System Server (storing information); monitoring system (processing information); an app installed on patients devices, as well as the iBeacons. Having reviewed the previous literature, we found that there is a lack of research in measuring patient experience through technology.  No study has measured patient experience through the use of technology in real time; except using an iPad for assessing patient experience at the end of the journey Benson & Potts (2014).  Therefore, we decided to devise a system for hospitals that can present location-based specific questionnaires to patients.

Materials and Methods

The main aim of this solution is to develop a location aware system, PJM system which consists of:  mobile application that delivers surveys based on the patient’s location and a website which calculate specific statistics from the retrieved information of these surveys. The intended users of the proposed system can be classified into three types:•  Admin: who is responsible of managing the system and the patent journey as well.•  Patient: who is responsible of answering the system surveys in their phone.•  Decision Maker: who is responsible to make decision regarding patents journey based on the calculated statistics from the retrieved information from surveys.The main features of the system are presented below according to the beneficiary user of the Service: •  Admin:  Admins manage the PJM through our system website (admin home page). the main page for the admin, there are seven choices: add patient and decision maker, which allows the administrator of the system to add a new patient or decision maker to the database; manage patient and decision maker, which allows the admin to edit, and delete patient/decision maker and their information; add PJM, which allows the admin to create a new patient journey map (PJM) and adds new touch point if he did not find it within existing PJM elements; manage PJM, which allows the admin to edit, delete and archive a PJM; and the final choice is edit admin profile. In add PJM, the admin can add survey and beacon ID by clicking on each touch point.  See Figures 1, 2, and 3.•  Decision maker: our system gives decision makers the ability to display patients’ feed- back in three types of charts which are pie, bar and stacked bar on the system website (decision maker home page). More specifically, the decision maker home page displays all PJMs to select particular PJM for viewing its feedback.  The system allows the decision maker to print a feedback. See Figure 4. Patient: Patients first log in to the mobile application using a unique identification (pa- tient ID and password) which is stored in the system database. After a successful login, a pop-up notification will be sent asking permission to enable Bluetooth connectivity on the patient’s mobile phone, and once this is enabled the device interacts with the BLE beacon signal and sends the iBeacon ID to the server.  Finally, the server sends the specific survey to the patient mobile. See Figures 5 and 6.

Figure 1: Admin Home Page

Figure 2: Add New Touch Point

Figure 3: Add a patient journey

Figure 4: Decision Maker’s Interface

Figure 5: Mobile App Interface

Figure 6: Beacon Detection

Figure 7: Sociotechnical approach for PJM system

We present the design and implementation processes of PJM system based on Socio-technical approach Sommerville (2010). Socio-technical approach in systems development pertains to theory regarding the social aspects of people and technical aspects of organizational structures.  The framework of the Socio-technical approach as suggested by Sommerville (2010) consists of several stages, which rely on gathering system requirements from the end-users of the system and considering the organization structure and administration processes.  The requirement also should consider the hardware in the deployment environment. Socio-technical approach assumes huge systems and complex environments. In our system, we have a simple environment which has several iBeacons installed along the patient’s journey path, a sever, and a few mobile phones connected to the server at a time. However, it is essential to capture the relation between the iBeacon, the patient, and the server.  For that, we choose the Socio-technical approach, but we adopt a simpler version, which focus on gathering requirements considering end users, hardware locations, and the patient journey surveys provided by the Saudi ministry of health, see Figure 7. Then, the following phases of analysing requirements, system implementation, deployment and the system testing will be considered. For system architecture, we adopt the client-server architecture as our system consists of mobile phones connected to iBeacons and website.  Our architecture including the main components, which are the iBeacon, mobile application for the patient, platform for admin/decision maker and system server. These components aggregate information regarding the location of the patient and provide content based on patient location. The iBeacon devices are deployed at each PJM item such as Reception, Waiting Room, Clinic and Pharmacy. When the patient enters the iBeacon range, they can receive and interpret radio signals from it.  The installed smartphone application sends the patient login information and the iBeacon information (iBeacon ID) to the server. The server determines the location of the user according to information from the pre-configured iBeacon, and then sends the specific survey, based on location, to the patient.

The application is developed to work under the Android platform.  The programming language used for developing the application is Java. In addition, MySQL is used as a database tool and PHP as the API to allow the Android application and website to communicate with the project server. JSON parser is used for data interchanging between application and server.

Figure 8: System Implementation

Results and Discussion

This section covers the process of testing our system. The principal objective of the testing process is to assess the system’s quality and to ensure that it does not contain bugs or errors A number of tests have been run, including both performance and acceptance testing.  All possible test cases and their results have been provided.

Performance Testing

The process of assessing the effectiveness or speed of a device, computer, network or piece of software is known as performance testing.  This may consist of conducting quantitative test in a laboratory, for example to determine the system’s response time or the number of millions of instructions per second that it works at. This process could also involve tests to assess qualitative attributes such as interoperability, reliability or scalability.  Stress testing is often carried out at the same time as a system’s performance is being assessed. We have tested the performance of our application by using the tool   SystemPanel on an Android mobile phone.  SystemPanel is an app that allows to view and manage device activity with simple visualizations SystemPanel (2019).•  CPU usage: The total CPU utilization that have is about 1.73% as a minimum value  when the application does not receive any data. On the other side, it can reach 3.20%.•  Memory usage:  The memory consumption of our application is 2.48 MB of the disk storage.  This means the application is acceptable compared with other applications. In fact, it is considered as one of the applications that uses the least memory storage.

User Acceptance Testing

User acceptance testing (UAT), sometimes known as End-User Testing, is one of the most important elements of the testing phase of any software development process. This is be- cause, throughout this phase, the actual users for whom the software is intended will test its functions. These tests are undertaken in order to ensure that the software can successfully execute the tasks and functions which may otherwise have been overlooked in real world scenarios. Therefore, this is an essential process and must be undertaken prior to the release of the software to the real-world market and its installation for the final clients Hambling & Goathem (2013).

Effectiveness, efficiency and satisfaction are the criteria which will be considered in order to measure the degree of usability. This will be based on the number of errors which occur for each function. This information serves to disclose the effectiveness of these functions. In addition, the time that the user takes to perform a specific function is recorded and quantified in order to measure the efficiency of a given function. Finally, at the conclusion of the tests, the end users will complete a questionnaire so that their impressions of the system may be considered Hambling & Goathem (2013).

Admin and Decision makers

In order to test the efficiency of the system, the average number of errors and the average length time that users took to complete the task were calculated and recorded for both the website and application users. Firstly, the groups and their functional tasks were defined. Following this, the acceptance tests were conducted with 6 users represented the admins and decision-makers in terms of their tasks on the website. The participants are aged between 20 to 35 years old. Tables 1 and 2 show the results for the user acceptance tests for the two users. A survey is made to measure the user satisfaction with the system. The survey results showed that the interfaces and menus are flexible, and the color and fonts are clear enough to most of the participants. Most of the participants found the navigation between web- site/application screens is easy and clear. They also indicated that accessing the system and learning what it offers is easy. The majority of the participants also indicated that finding the commands required to complete a task is easy and the options names made sense to them. In general, they think that the functions provided by the system are efficient and effective. The majority of the participants find the procedure of the function is simple and require a minimum number of steps and the time required to complete it is reasonable. All of the participants agree about using the system in the future.

Table 1: User Acceptance Tasks and Results for the Admin

Task	Number of errors (Average)	Time in Seconds (Average)	Results
Login	0	11.37	pass
Add a new patient	0	25.5	pass
Delete a patient	0	14.6	pass
Add a new PJM	0	37.4	pass
Edit PJM	0.33	50.25	pass
Logout	0	3.08	pass

Table 2: User Acceptance Tasks and Results for The Decision Maker

Task	Number of errors (Average)	Time in Seconds (Average)	Results
Login	0	10.22	pass
view	0	2.22	pass
Print	0	3	pass
Logout	0	2	pass

We have conducted an experiment at Qassim Regional Dental Center in Buraydah to evaluate the proposed application from the patient’s point of view. To enable the measurement of patient experience in outpatient clinics at the Ministry of Health in the Kingdom of Saudi Arabia, 6 touch points have been identified. These include: appointment booking, location information and service availability, registration and waiting, patient mobility and facility environment, medical services, support services such as the laboratory, radiology and pharmacy procedures. Two touch points were excluded, medical services and support services. For medical services, some patients will spend far longer than an hour at the clinic, for treatments like dental implants, and as it is not required by all patients, support services were also excluded. Three beacons were deployed at different locations, in the reception, waiting room and hallway. Participants The total number of participants in the experiment was twelve, three were male and 9 were female. The participants were aged between 20-50 years old with the majority of them in their 40?s. Half of the participants held a bachelor’s degree or higher, and the other half possessed a high school diploma. All of the participants have experience with using smartphone applications.

A test plan comprised of three stages was developed. Stage 1 introduction: includes welcoming participants and providing a brief description of the app with verbal instructions for the testing procedure. Then, the participants were asked to complete a demographic questionnaire to collect background data regarding their gender, age, and experience in using smartphone apps. Stage 2 app testing: we sought to test the technical effectiveness of the app (i.e., whether or not the user could complete a given task). To achieve this, participants were given 5 tasks to complete (Table 3 ). First, we asked the participants to log in then respond to the app notification to evaluate the service using a five-point rating scale (Likert scale). And so on, for every touch point in their journey. Stage 3 user satisfaction: we examined the user’s satisfaction with the app by completing the PJM app experience questionnaire. Prior to testing, the PJM app was installed on an Android mobile (HTC U Play). The app was tested to ensure that it had downloaded correctly, was functioning without error, and was connected to a WiFi network. All tasks were completed successfully with the users commenting on how easy the app was to use. However, two participants did record experiencing difficulties with the apps logout process. The participants commented that they had to return to the main interface to logout. This is because the user is required to answer all touch point questions, when this is done the return button will become available.

Table 3: Description of Tasks for the Patient

Task number	Task Name	Task Description
Task 1	Login	Login with patient’s ID and password
Task 2	Evaluate Location information and reception services (appointments booking)	Score the service based on the five points rating scale.
Task 3	Evaluate Registration and waiting	Score the service based on the five points rating scale.
Task 4	Evaluate Patient mobility and facility environment	Score the service based on the five points rating scale.
Task 5	Logout	Terminate the session.

The user satisfaction survey revealed that all participants rated the app as being important and easy to use. Moreover, most of the participants stated that they were willing to use the app again. There were two recommendations raised by the users to improve the app. First, the colours used needed to be brighter and second, the text should be larger (Figure 9). Patients reported that they strongly agree that the application is a suitable replacement for paper.

Figure 9: Patient satisfaction survey results

Conclusion

In this paper, we proposed a system that uses a Bluetooth Low Energy based iBeacon for measure patient experience.  Patient can assess healthcare services via Android devices in real time. This reduces the time needed and produces more accurate results compared to the traditional hand written approaches. The intended users of the proposed system are classified into three types: admin, decision maker and patient. For admin, a platform that allows the patient experience admin to create and manage a new patient journey map is developed. In addition, decision maker is able to retrieve feedback from patients in a visual way to identify deficiencies in the service and speed of processing. For the patient, a mobile application that allows them to complete a survey based on their location is developed.  After developing the proposed system, the usability is measured using three criteria:  effectiveness, efficiency and satisfaction.   The results showed that the system gave satisfactory performance and achieved the usability requirements. The system can be improved in future by implementing it on other operating system like iOS. The same technology can be used in various other applications such as banking services, museum, restaurants services, retail stores etc.

Acknowledgements

This research project was supported by a grant from the “Research Center of the Female Scientific and Medical Colleges Deanship of Scientific Research, King Saud University.

References

Ahmed, F., Burt, J. & Roland, M. (2014), Measuring Patient Experience: Concepts and Methods, The Patient – Patient-Centered Outcomes Research 7(3), 235–241.

Bate, P. & Robert, G. (2006), Experience-based design: from redesigning the system around the patient to co-designing services with the patient, Quality & Safety in Health Care 15(5), 307–310.

Benson, T. & Potts, H. W. (2014), A short generic patient experience questionnaire: howRwe development and validation, BMC Health Services Research 14(1), 499.

Bessant, J. & Maher, L. (2009), Developing radical service innovations in healthcare the role of design methods, International Journal of Innovation Management 13(04), 555–568.

Bjertnaes, O. A., Sjetne, I. S. & Iversen, H. H. (2012), Overall patient satisfaction with hospitals: effects of patient-reported experiences and fulfilment of expectations, BMJ quality & safety 21(1), 39–46.

Doyle, C., Lennox, L. & Bell, D. (2013), A systematic review of evidence on the links between patient experience and clinical safety and effectiveness, BMJ Open 3(1), e001570.

Druckenmiller, G. (2016), What is the Patient Journey and Why is it Important? | Evariant, https://www.evariant.com/blog/why-is-patient-engagement-journey-important.

Floyd, R. W. (1962), Algorithm 97: Shortest Path, Commun. ACM 5(6), 345.

Hambling, B. & Goathem, P. V. (2013), User Acceptance Testing: A Step-By-Step Guide, BCS, the Chartered Institute for IT, Swindon.

He, Z., Cui, B., Zhou, W. & Yokoi, S. (2015), A proposal of interaction system between visitor and collection in museum hall by iBeacon, in ‘2015 10th International Conference on Computer Science Education (ICCSE)’, pp. 427–430.

Institute, T. B. (2019), Defining Patient Experience – The Beryl Institute – Improving the Patient Experience, https://www.theberylinstitute.org/page/DefiningPatientExp.

Jenkinson, C., Coulter, A. & Bruster, S. (2002), The Picker Patient Experience Questionnaire: development and validation using data from in-patient surveys in five countries, International Journal for Quality in Health Care: Journal of the International Society for Quality in Health Care 14(5), 353–358.

Lin, X., Ho, T., Fang, C., Yen, Z., Yang, B. & Lai, F. (2015), A mobile indoor positioning system based on ibeacon technology, in ‘2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)’.

M.D, J. M. & Lundeen, T. (2015), Service Fanatics: How to Build Superior Patient Experience the Cleveland Clinic Way, unabridged edition edn, McGraw-Hill Education on Brilliance Audio.

Robert, G. & Cornwell, J. (2013), Rethinking policy approaches to measuring and improving patient experience, Journal of Health Services Research & Policy 18(2), 67–69.

Sommerville, I. (2010), Software Engineering, 9 edition edn, Pearson, Boston.

Stull, D. E., Leidy, N. K., Parasuraman, B. & Chassany, O. (2009), Optimal recall periods for patient-reported outcomes: challenges and potential solutions, Current Medical Research and Opinion 25(4), 929–942.

SystemPanel (2019), SystemPanel 2 – Apps on Google Play, https://play.google.com/store/ apps/details?id=nextapp.sp&hl=en.

Trebble, T. M., Hansi, N., Hydes, T., Smith, M. A. & Baker, M. (2010), ‘Process mapping the patient journey: an introduction’, BMJ 341, c4078.

Trebble, T. M. & Hydes, T. (2011), Redesigning services around patients and their doctors: the continuing relevance of lean thinking transformation, Clinical Medicine 11(4), 308–310.

Wheeler, S. M., Gilbert, J. E. & Klonikowski, E. (2015), The patient patient: The importance of knowing your navigator.

Wolf, J., Niederhauser, V., Marshburn, D. & LaVela, S. (2014), Defining Patient Experience, Patient Experience Journal 1(1), 7–19.

Yang, J., Wang, Z. & Zhang, X. (2015), An ibeacon-based indoor positioning systems for hospitals.