Information Technology Department, College of Computer and Information Sciences, King Saud University, Riyadh, Saudi Arabia

Corresponding author email: ralabduljabbar@ksu.edu.sa

Article Publishing History

INTRODUCTION

IT knowledge and skills are critical elements for nations to prosper and compete. Therefore, the quality of education and ensuring that graduates possess sufficient knowledge and skills to meet the demands of the market are necessary. One of the main obstacles is that many of the graduates are not equipped with industry-relevant skills, particularly in industries related to science and technology. In Saudi Arabia, recent research shows that Saudi’s technological sector exhibits skills gap in two primary areas: technology-based skills and soft skills.

It is important to note that many of the technology-based skills are taught at college or university level (Mishrif and Alabduljabbar 2018). Also, in Saudi Arabia, there is a lack of coordination between institutions of higher education and the world of work (Ibeaheem, Ragmoun, and Elawady 2017). In general, most employers believe that graduates from the universities are not unequipped with the required soft skills (Kennedy 2019).

Saudi Arabia is one of the top spenders in the information and communications technology (MCIT 2014). Such a massive spending leads to the creation of a large number of IT jobs. Conversely, one of the real challenges of the 21st century in Saudi Arabia is the high rate of unemployment. In 2020, the unemployment rate is increasing according to the statistics from the Saudi labor market. One of the causes behind this high rate is the gap between the education outputs and the actual requirements of the Saudi labor market (Labor Market Reports 2020).

Previous studies have reported the use of prediction models in employment. Some models are used to match the right talent to the right job (Zhu et al. 2018), whereas others predict employee turnover (Bhulai 2016).  Another predictive model was built for predicting a career success in engineering among women and African American men (Charitable 2011). Also, another research (Paparrizos, Cambazoglu, and Gionis 2011) studied the problem of recommending jobs to people who are seeking a new job. They developed a supervised learning model that predicts the next job transition of a person and recommends jobs to people.

Another study (Al-Dossari et al. 2020) aimed to compare the extreme gradient boosting (XGBoost) technique against five models to help IT graduates select a career path based on their skills. A similar study (Punnoose and Ajit 2016) aimed to compare the XGBoost against six supervised models using the data from HR Information Systems (HRIS) to predict employee’s turnover. Results show that (XGBoost) provided significantly higher accuracy in predicting the employee turnover.As information about the requirements of the labor market in Saudi Arabia is available on online job portals and the current education outputs can be extracted from the users, we can match the demands of the Saudi labor market to solve the problem of the high rate of unemployment by knowing the disequilibrium among current education outputs, particularly in IT jobs.

The work presented herein has the following contributions that distinguish it from other works in this area: The first contribution is the development of a prediction model that can be deployed on the web, where users can input variables to get predictions on their suitability for recruitment in the Saudi IT labor market. The model can generate predictions directly from the source, that is, by using data extracted from two online job portals, namely, LinkedIn and Bayt.com, as posted by employers in their job announcements. The second contribution is providing data-driven recommendations of the in-demand skills in the IT labor market in Saudi Arabia to overcome the unemployment problem.

MATERIAL AND METHODS

• This work seeks to address the following research questions:
• What are the most in-demand skills required for recruitment in the IT Saudi labor market?
• How can we predict the suitability of an IT graduate for recruitment?
• How can we reduce the perceived skills gap and enhance the employability of IT graduates?

The first research question has been addressed by collecting and analyzing data from online job portals to better understand the in-demand skills and jobs. To resolve the second research question, a machine-learning algorithm was utilized to build the suitability prediction model and a web-based application was developed to interact with the model based on the model. Finally, to evaluate the existing gap in education, senior and graduate IT students were surveyed to determine whether they possess the required skills as predicted from our analysis on job advertisements on LinkedIn and Bayt. The overall solution can be seen in Figure 1.

Figure 1: Methodology

Data collection: Data were collected from two online job portals, LinkedIn and Bayt.com. Data were collected over a three-month period (October 2018–December 2018) through web scraping using the OCTOPARSE tool (Octoparse 2020). OCTOPARSE is an automatic powerful tool used to extract data from web pages. Overall, the total number of job posts extracted was 675 job requests with each job request having its own features, e.g., title, description, and required skills.

Subsequently, a content analysis was performed on these job requests as a first step to determine the most requested skills within these job requests. Content analysis is a widely used qualitative research technique (Krippendorff 2018). Each job request was processed by the tool (Octoparse 2020) as a separate case. The resulted data set comprised 170 skills.

Data cleaning: To meaningfully label the dataset, preprocessing of the dataset was accomplished by applying some filters using the OpenRefine tool (OpenRefine 2020) in the following order:

Remove unwanted characters (nonalphabetic characters) using the “expression value” option in the Open Refine. Stop-words removal filter: Stop words are words that are filtered out before the processing of textual data. Some of the common stop words are ‘‘the, is, at, but be, been, and, as, out, ever, own, he, she, and an”. Filtering/Faceting Data: It is a method to filter data into subsets for ease of use that can be done for text, number, and dates. (FacetàText Filter). Lowercase filter: Keywords are converted to lowercase letters to simplify the comparison. After the cleaning process, the dataset was pared down to 120 skills.

Data labeling: To classify the data set into two classes, “wanted” and “unwanted” skills, the average of the occurrences of skills was used as a determination point. Any skill with an occurrences number above the average was considered as a “wanted” skill, whereas any skill with an occurrences number below the average was considered as an “unwanted” skill. After experimenting, the average was calculated using the median formula because it is the most suitable measure of average for data classified on an ordinal scale. Also, the median formula is a good measure of the average value when the data include exceptionally high or low values.

To better understand in-demand skills for recruitment in the IT Saudi labor market, the related skills are grouped into categories. Skills categorization was created by examining the Association for Computing Machinery (ACM) IT Curriculum (ACM 2017) and other empirical study (El-Gabaly and Majidi 2003). Accordingly, skills were categorized into hard skills, soft skills, programming languages, study degree, certificates, and languages. Subsequently, a list of 76 hard skills, 21 soft skills, 13 programming languages, three study degrees, six certificates, and two spoken languages’ skills were identified. Figure 2 provides a summary of the in-demand skills in each skill category.

Figure 2: In-Demand Skills in Each Skill Category

Among the hard skills shown in Figure 2 (a), the development and project management skills are the most required skills, whereas among the soft skills listed in Figure 2 (b), communication and teamwork skills are the most required. Figure 2 (c) shows that the most required certifications in IT are CISCO and ITIL, whereas Figure 2 (d) shows that the most required educational degree in IT jobs is a bachelor’s degree. Moreover, Figure 2 (e) shows that the most in-demand programming languages are JavaScript and Java, whereas Figure 2 (f) shows that most IT jobs require fluency in English.

Implementation: Normally, supervised learning techniques are used in classification models. Among the several algorithms proposed for the supervised classifications of texts, Support Vector Machine (SVM), Naïve Bayes (NB), and k-Nearest Neighbor (k-NN) were used in this study because these algorithms were shown to be the most appropriate in the existing literature (Khan et al. 2010). The resulting dataset was used as a training sample for the classifier to automatically detect the class of unlabeled skills. With this small data size, it is recommended to use n-fold cross-validation and percentage split technique (Brownlee 2018). For the percentage split technique, 70% of the data is used to train the model and 30% is used for testing.

The classification was conducted using the Waikato Environment for Knowledge Analysis (Weka) toolkit (Waikato 2016). Before deciding on which classification algorithm to use, an experiment was conducted with other classifiers, namely, NB and k-NN. Figure. 3 shows a comparison between the three classifiers using the10-fold cross-validation.

Figure 3: Comparison of Classifiers using the 10-Fold Cross-Validation

Comparison results show that SVM achieved a good performance in comparison with the k-NN and NB classifiers using the 10-fold cross validation. However, determining the accuracy alone is insufficient, as sometimes, having high accuracy values does not necessarily imply that the algorithm has an excellent performance. Hence, it is necessary to look at other aspects to decide which algorithm is the best. Percentage split (70%) validation technique was used to ensure that the model performs well with different test data in the future. Figure 4 shows a comparison between the three classifiers using percentage split validation.

Figure 4: Comparison of Classifiers using Percentage Split

Results from both comparisons (Figure 3 and Figure 4) showed that SVM was the best classifier technique. With percentage split option, K-NN and SVM yielded the same accuracy, which was 69.44%; however, ROC area in SVM was better than that in K-NN. Hence, SVM was the best classifier with the highest accuracy. To enhance the model performance, domain experts were consulted who came up with four options that can be opted in the study. The options include increasing the dataset size, performing feature selection, changing classifier parameter, and using ensemble methods (Bagging and Adaboots).

Increasing the size of the data takes a long time, which is not desired herein. The feature selection option has been eliminated as well because it is not recommended to apply attribute selection to all of the data and then run an evaluation on the dimensionally reduced data. Doing this will give overly optimistic error rates because the attribute selection process has used the data from the test folds. Conversely, ensemble methods do not work perfectly with all the three classifiers. This leaves one available option, which is changing the classifiers parameter. In the SVM classifier, after changing the kernel function from PolyKerne to PUK, the accuracy increased to 72.22%, whereas no increase in the accuracy was observed in K-NN and Naïve base after changing the parameter.

The web application was created so that the users can interact with the prediction model, i.e., the users can input their skills to generate predictions about their suitability of recruitment. Figure 5 shows the web application interface and its different components. The interface was designed to be user-friendly, simple, and consistent. As shown in Figure 5, the required skills are presented to the users. If the user fills in the form on the page and clicks on a “next” button, the application extracts the input, runs it through the model, and will finally render result.html with the results in place. The user can then view two items in the results page: 1. User’s suitability for recruitment and 2. Recommendations to help the user addresses her skills shortage.

Figure 5: Web Application Interface: (1) User’s Skills Selecting Page; (2) Next Button to the Results Page; (3) User’s Suitability to Recruitment in the IT Saudi Labor Market; (4) Recommendations to Help the User Develop her Skills; And (5) Back Button to the Skills Page.

Descriptive and data analysis methods were employed in this study to evaluate the existing gap in education. Senior and graduate IT students in Saudi Arabia were surveyed to determine whether they possess the required skills as predicted from our market analysis of the online job portals. To ensure that the questions were clear, complete, and unambiguous, experts were asked to participate in a pilot study to validate the content and style of the survey.

Feedback was gathered, leading to an improved version of the survey that was then distributed to the participants. The final survey comprised 44 skills and was administered via a commercial online survey tool. The survey instrument comprised five main sections: demographic information, hard skills, soft skills, programming languages, and specialized technical certificates. Participants were asked to rank themselves in terms of importance on a scale of 1 (no level of competence) to 5 (high level of competence).

Incorporating Rating Scale questions, weights can be assigned to each answer option. This will enable to calculate the weighted average for each answer option. On the scale, the answer with a high score or high average is the most preferred one. The weighted average is calculated as follows:

With Likert and Likert-like survey questions incorporating numbers as options, it is easier for participants to provide their answers and for researchers to analyze results. In relation to our data, any measured skill scored a weighted average above than three (the midpoint of the scale 1–5) can be considered as gained by the participant, whereas a value below three would indicate lacking this skill (Boone and Boone 2012).Responses were received from more than 100 participants, 72% of which were females and 28% were males. 79% of the respondents surveyed were senior IT students and 21% were IT graduates. The participants belong to different universities in Saudi Arabia, namely, King Saud University, Imam Mohammad University, Prince Nora University, and Prince Sultan University.

RESULTS AND DISCUSSION

Tables 1, 2, and 3 present the results collected from the students, which show their self-assessment of hard skills, soft skills, and programming languages, respectively, that they are either holding or lacking. Skills possessed by a participant with a weighted average greater than three are shown in bold. Results show that the surveyed students claimed that they are lacking some of the required hard skills such as image and video processing and Unix Linux platforms, whereas they claim mastering the development, research, and analysis skills. Furthermore, all surveyed students claim that they have all of the soft skills. Regarding the programming languages, the students claim that they lack knowledge in crucial programming languages such as C# and Python. Also, only 21% of the respondents hold IT certifications.

The first set of findings herein indicates that the top in-demand hard skills (Figure 2 (a)) by the Saudi IT market are development skills, project management, and analysis which are the same as the top-possessed skills by the senior IT and graduates survey (Table 1). In addition, the survey of IT graduates shows that mastering or familiarity with Microsoft Office is on the top of the acquired skills by an IT graduate; however, it was not at the top of the in-demand skills required by the market. This might be because as an IT graduate, employers assume that this is a fundamental requirement and a must-have skill that should not be explicitly specified in the job requirement description. Moreover, according to our skill demand analysis, application skills in security and network are the top in-demand skills by the Saudi IT market.

This is also consistent with a recent finding from (Al-Khalifa 2017), which indicated that Network and Security are two areas that are in demand in the current market. Conversely, these skills scored below average in our IT graduates survey. There have been recent developments in skills teaching and technical learning in curriculum within local universities over the years. This development aims at providing the required skills within the labor market. There is a notable change and room for improvement, particularly within network and security applications.

The second finding of this study is related with soft skills. Our analysis of market requirements (Figure 2 (b)) and senior IT and graduates survey (Table 2) suggests positive results because the top in-demand skills (communication and teamwork skills) are possessed by senior IT and graduates. However, we believe that this may not be true. Students who have graduated from a university program assume that they are good communicators and strong team players. According to (Stevens and Norman 2016), who surveyed and interviewed 12 employers in the local IT market, graduates from the universities are unequipped with the required soft skills.

For certifications, the top in-demand certificate in the IT Saudi Market (Figure 2 (c)) is Cisco. However, only 10% of the senior and IT graduates have it as revealed by the survey results. Furthermore, our findings show that only 21% of respondents hold IT certificates and 61.90% of those are male. Looking closely, it can be concluded that certifications including Cisco, ITIL, and PMP will add value to the resume of the job titles as they are high in demand in the IT Saudi labor Market. Another interesting finding suggests Python, C#, and JavaScript as the most in-demand programming languages in the IT Saudi Market (Figure 2 (e)).

However, these were pointed as major areas of weakness in the possessed skills by IT graduates according to the survey conducted (Table 3). This could be because unlike Java, Python and C# are not offered as separate courses in educational institutions. In addition, from the most in-demand programming languages, Java, CSS, and HTML are possessed by senior IT and graduates. One interesting finding in the survey is that MySQL scored high in the possessed skills by IT graduates, whereas it is not in-demand in the market.

Table 1. Hard Skills Possessed/Lacked By Participants

	(1) no level of competence	(2) low level of competence	(3) moderate level of competence	(4) moderately high level of competence	(5) high level of competence	Weighted average
Microsoft office	0%	9%	16%	24%	51%	4.17
Technical skills	2%	28%	28%	28%	14%	3.22
Development skills	2%	25%	32%	30%	11%	3.21
Research skills	5%	28%	25%	24%	18%	3.17
Project Management	2%	33%	28%	21%	16%	3.14
Analysis skills	5%	31%	21%	30%	13%	3.10
Software engineering	8%	32%	21%	30%	9%	2.92
Database management	2%	40%	30%	18%	10%	2.92
Technical support	8%	33%	27%	22%	10%	2.85
Designing skills	5%	44%	20%	25%	6%	2.78
Security	7%	35%	36%	14%	8%	2.74
Systems Administrator	9%	34%	34%	17%	6%	2.68
Networking	6%	43%	28%	18%	5%	2.67
Oracle	11%	39%	27%	13%	10%	2.61
Maintenance skills	15%	44%	29%	8%	4%	2.27
Data mining	25%	38%	12%	18%	7%	2.19
Image & video processing	23%	41%	18%	11%	7%	2.15
Unix Linux platforms	20%	54%	18%	5%	3%	1.97

Table 2. Soft skills possessed/lacked by participants

	(1) no level of competence	(2) low level of competence	(3) moderate level of competence	(4) moderately high level of competence	(5) high level of competence	Weighted average
Teamwork skills	0%	3%	26%	31%	40%	4.05
Responsibility skills	1%	4%	31%	17%	47%	4.00
Self-learning	1%	2%	26%	38%	33%	3.97
Communication skills	0%	4%	32%	33%	31%	3.87
Planning skills	0%	3%	36%	35%	26%	3.81
Leadership	1%	9%	34%	21%	35%	3.70
Coordinating skills	2%	3%	39%	32%	24%	3.68
Creative thinking	0%	3%	45%	32%	20%	3.66
Time management	3%	9%	38%	32%	18%	3.41

Table 3. Programming languages possessed/lacked by participant

	(1) no level of competence	(2) low level of competence	(3) moderate level of competence	(4) moderately high level of competence	(5) high level of competence	Weighted average
Java	0%	13%	28%	32%	27%	3.73
Html	4%	22%	14%	26%	34%	3.6
Html5	11%	22%	11%	30%	26%	3.27
MySQL	6%	24%	29%	26%	15%	3.14
CSS	12%	21%	18%	29%	20%	3.12
JavaScript	9%	34%	23%	25%	9%	2.82
C#	33%	27%	14%	19%	7%	2.07
Python	41%	29%	17%	10%	3%	1.64
ASP.net	51%	23%	11%	10%	5%	1.44

Results showed that there existed a gap between labor market employers’ expectation of Saudi workers and the skills that the workers were equipped with from their educational institutions. Moreover, the result explains that the educational output has been improved over the past years, particularly in the soft skills. The gap existing between the required skills and the skills offered affects the operation of organizations and calls for upskilled employees to act as an antidote to the collapsing organizations. Certification is a proven way to enhance advanced critical skills that are required by the IT department in their operations. Our findings are also consistent with other previous studies such as (Al-Khalifa 2017) and (Al-Dossari et al. 2020) which indicate that new IT market requires professionals with international certificates and mastery of specific programming languages.

A limitation in this study stems a relatively high percentage (72%) of respondents being female. Conversely, the second limitation in this study stems from a relatively low number of respondents belonging to private universities. To curb these limitations, the research project may consider using more male respondents from more private universities so as to expand the research scope. In addition, another limitation in this study regarding the prediction model is the size of the training data set, which might affect the classification accuracy. However, as explained previously, we consulted domain experts to improve the accuracy.

CONCLUSION AND FUTURE WORK

This study aimed to identify in-demand skills for recruitment in the Saudi technological sector in-order to allow graduates to predict their suitability for recruitment. Results showed that there existed a gap between employers’ expectation of graduates and the skills that the graduates were equipped with. Planned collaboration between industry and education institutions is required to narrow down this gap by providing courses that address these skills.  Furthermore, educational institutions should offer academic credits for the completion of IT certifications. The study has  left a scope for future work to investigate the implications of COVID-19 on the recruitment in the technological sector.

ACKNOWLEDGEMENTS

The authors would like to thank the Deanship of Scientific Research and RSSU at King Saud University for their technical support.

Declaration of Interests: The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

REFERENCES

ACM (2017) Task Group on Information Technology Curricula. Information Technology Curricula 2017: Curriculum Guidelines for Baccalaureate Degree Programs in Information Technology. New York, NY, USA: Association for Computing Machinery

Al-Dossari, H., Nughaymish, F.A., Al-Qahtani, Z., Alkahlifah, M., and Alqahtani, A. (2020) A Machine Learning Approach to Career Path Choice for Information Technology Graduates. Engineering, Technology & Applied Science Research 10 (6), 6589–6596

Al-Khalifa, H. (2017) A Survey of IT Jobs in the Kingdom of Saudi Arabia 2017 [online] available from <https://www.slideshare.net/hend.alkhalifa/a-survey-of-it-jobs-in-the-kingdom-of-saudi-arabia-2017> [10 December 2020]

Bhulai, S. (2016) Analysing Which Factors Are of Influence in Predicting the Employee Turnover.

Boone, H.N. and Boone, D.A. (2012) Analyzing Likert Data. Journal of Extension 50(2),1-5

Brownlee, J. (2018) A Gentle Introduction to K-Fold Cross-Validation [online] available from <https://machinelearningmastery.com/k-fold-cross-validation/> [10 December 2020]

Charitable, R. (2011) A Model for Predicting a Career Success in Engineering Among Women and African American Men. ProQuest Dissertations and Theses

El-Gabaly, M. and Majidi, M. (2003) The Information Communication Technology (ICT) Penetration and Skills Gap Analysis (SGA). Planning and Learning Inc.

Ibeaheem, H., Ragmoun, W., and Elawady, S. (2017) The Role of Saudi Universities on the Improvement of Higher Education Skills on Saudi Arabia. The Business and Management Review 9 (2), 69-82

Kennedy, H. (2019) The Labor Market in Saudi Arabia: Background, Areas of Progress, and Insights for the Future. Harvard Kennedy School

Khan, A., Baharudin, B., Lee, L., and Khan, K. (2010) A Review of Machine Learning Algorithms for Text-Documents Classification. Journal of Advances In Information Technology 1 (1), 4-20

Krippendorff, K. (2018) Content Analysis: An Introduction to Its Methodology. SAGE Publications, Inc

Labor Market Reports (2020) Jadwa Investment [online] available from <available: http://www.jadwa.com/en/researchsection/research/economic-research/labor-market-reports> [9 June 2020]

MCIT (2014) IDC: Saudi IT Spending to Reach $14.2bn In 2017 [online] available from <available: https://www.mcit.gov.sa/en/media-center/news/92185> [3 July 2020]

Mishrif, A. and Alabduljabbar, A. (2018) Quality of Education and Labour Market in Saudi Arabia. In: Mishrif A., Al Balushi Y. (Eds) Economic Diversification in the Gulf Region 1. The Political Economy of the Middle East. Palgrave Macmillan, Singapore

Octoparse (2020) Octoparse Tool [online] available from <https://www.octoparse.com/> [10 November 2020]

OpenRefine (2020) OpenRefine Tool [online] available from <https://openrefine.org/> [10 October 2020]

Paparrizos, I., Cambazoglu, B., and Gionis, A. (2011) Machine Learned Job Recommendation. in The Fifth ACM Conference on Recommender Systems. Association for Computing Machinery, New York, NY, USA. held 2011. 325–328

Punnoose, R. and Ajit, P. (2016) Prediction of Employee Turnover in Organizations Using Machine Learning Algorithms. International Journal of Advanced Research in Artificial Intelligence 5(9), 22-26

Stevens, M. and Norman, R. (2016) Industry Expectations of Soft Skills in IT Graduates: A Regional Survey. in The Australasian Computer Science Week Multiconference. Association for Computing Machinery, New York, NY, USA, Article 13, 1–9. held 2016

Waikato (2016) Weka 3 – Data Mining with Open-Source Machine Learning Software in Java. in The University of Waikato.

Zhu, C., Zhu, H., Xiong, H., Ma, C., Xie, F., Pengliang, D., and Li, P. (2018) Person-Job Fit: Adapting the Right Talent for the Right Job with Joint Representation Learning. ACM Transactions on Management Information Systems 9 (3), 17 pages