Characterizing business intelligence tasks, use and users in the workplace

This paper investigates business intelligence (BI) tasks, use and users in a workplace setting. The study reports on a mixed methods study of users in three different types of organisations employing BI. 1052 respondents answered a survey and 15 individual and 3 group interviews were conducted to elaborate on the survey results. The study finds that the majority of public BI users are employees, and fewer managers and students, that are handling a variety of tasks. Although they can experience challenges learning and using the BI system, they are still satisfied with it from different perspectives.


INTRODUCTION
In 2017, Gartner performed a worldwide survey of IT spending among 2500 chief information officers (CIOs). Business intelligence (BI) was one of the top technology priorities they identified ("Gartner Survey of More Than 2,500 CIOs Charts the Rise of the Digital Ecosystem," n.d.). One of the reasons for this focus on BI can be attributed to the increasing importance of BI systems. BI can be defined as "a broad category of technologies, applications, and processes for gathering, storing, accessing, and analysing data to help its users make better decisions" (Wixom & Watson, 2010, p. 13). In recent years, BI technologies have received considerable attention from both industry and the public sector (Chen et al. 2012). BI is an interesting technology because several studies have shown that there is a relationship between computerdriven decisions and organisational performance (Brynjolfsson, Hitt, & Kim, 2011). However, achieving BI success depends on both organization and staff characteristics (Worley et al. 2005;Salmasi et al. 2016). Salmasi et al. (2016) previously conducted a study of organisational level competences in achieving success with BI. However, García and Pinzón (2017) found that amongst others, the human perspective along with learning and skills are highly important to success. Therefore, we will focus on the individual perspective and focus on the users in this paper.
At the same time as the development of government processes, organisations and technologies are expected to change government employees' tasks. Before the emergence of e-government, governments' information technology and data management tasks were largely related to employment (see Kraemer & Dedrick, 1997). Today, changes in work tasks are an expected consequence of governments' digitising efforts. In particular, e-government is expected to affect the composition of public employees' tasks (Dörfler, 2003;Snellen, 2002). Jürgensen (2012) documented employees' expectations within the framework of administrative grants and found that specific, routine tasks had fallen from employees' daily tasks while the proportion of challenging applications had risen. Others note similar findings regarding tax department employees.
The present paper is concerned with characterising the tasks users solve with BI. Thus, to achieve success with BI at the user level, there is a relationship between task characteristics and success (Petter et al. 2013). Furthermore, we examine BI systems' ability to underpin the tasks their users solve with BI in a Danish e-government setting. The use of BI in e-government has spread worldwide in the latest decade. By improving access to BI among employees, governments are aiming to improve their decision-making processes, resource use, increase quality of the services delivered or even reduce costs.
The paper is structured as follows: In the following section, we review existing studies of task characteristics and present the theoretical framework for the data collection process. The next section is a presentation of the research methods applied in the study: a survey questionnaire, 12 semi-structured interviews and three group interviews. The subsequent section presents our findings with regard to the research question. The paper concludes with closing remarks and suggestions for further research.

The concept of a task
A task is what individuals engage in to keep their work or life continuing (Li & Belkin, 2008), and the concept of 'tasks' is important in human-computer interaction. A task (whether work-or leisure-related) may trigger information-oriented activities (Byström Katriina & Hansen Preben, 2005). On this basis, the task becomes a central element of any user's context, as it arises from an incident external to the user that first triggers an information need, followed by a searching activity (Ingwersen & Järvelin, 2005). Being external to the user, the task, as such, will be easier to observe and measure, from a research perspective, if compared to information needs that are inherent to the user.
Tasks have been analysed in humancomputer interactions from many different perspectives. Historically, the focus on tasks went from a technical (ergonomic) perspective, to a conceptual (information processing) perspective and then to work-process (contextual) models (Crystal & Ellington, 2004). Different approaches can be followed to gain insight into tasks in specific contexts.
Hierarchical task analysis breaks generic tasks into smaller sub tasks with related sub goals. (Stanton, 2006). The purpose is to become able to map goals, and sub goals in particular, with technologies or information systems to ensure successful solutions for the users' tasks.
A different way of perceiving tasks is to model them according to Li and Belkin's (Li & Belkin, 2008) taxonomy of task characteristics. Departing from a literature review, the taxonomy defines tasks on the basis of generic facets and common attributes, thus representing a top-down perspective on the task concept.
The different approaches to understanding and operationalising tasks emphasises the importance of the concept in human-computer interaction. We have not identified any papers to date within the BI systems field that have attempted to identify and characterise the specific tasks users carry out. The purpose of the current paper is to address this gap in the research in a public organisation context.

DeLone & McLean: The IS Success Model
DeLone and McLean's IS success model (DeLone & McLean, 1992) is used to frame the study. The model represents a framework for understanding influential factors on information systems' success. The identified variables in the model include system quality, information quality, use, user satisfaction, individual impact and organisational impact. We use the model to frame the quantitative data collection below, as it represents a consolidated theoretical model (eg. Iivari, 2005), providing both an organisational and system-based perspective on the notion of tasks.

RESEARCH DESIGN AND METHOD
In this study, we used a multiphase 'mixed methods' research design. The research design consists of two main phases, namely a questionnaire and interviews. The mixed methods approach represents a form of triangulation; the quantitative approach provides a broader view, while the qualitative approach provides greater depth. Together, the approaches yield results from which more accurate inferences can be made (Seddon et al. 1999).

Quantitative method
We chose a questionnaire to research users' perceptions of different task characteristics.
Data were collected via an online survey available for a specific period during the spring of 2017. All respondents were BI end-users who had access rights to their organisation's BI system. The users accessed the BI web client through a browser, meaning that BI can be implemented across an entire organisation without having to install software on each machine. All BI users from three public organisations were invited to complete the survey. The three organisations were a municipality using Business Objects, a public healthcare organisation (among 12 hospitals) using Tableau and a university using QlikView.
Initially, we conducted a pilot study before distributing the survey to all invitees. The survey was based on a literature review, and three researchers in the field evaluated the questions. Afterwards, BI users with differing levels of BI experience evaluated the questionnaire using a think-aloud test (Nielsen, 1994). Minor refinements were made based on these results. The final part of the pilot study called for testing the survey on 24 BI users. After evaluating those results, the questionnaire was distributed by email to 4901 invitees.
Participants accessed the questionnaire via a personal invitation email with a unique link to the online survey. Each respondent received an adapted questionnaire depending on whether he or she had previously used or never used the BI in question. Participation in the survey was voluntary, and two reminders were sent. In total, 1741 people completed the survey, resulting in a response rate of 35.52%. Among these, 1052 were used for the statistical analysis, as 689 respondents indicated that they did not use BI. All data were analysed in SPSS version 24.0.

Qualitative method
The next step in our research design was interviewing BI users. In addition to the questionnaire, we used interviews for three reasons: qualitative data can explain the complexity of the users' tasks identified in the survey, data from interviews helps us to grasp the users' contexts and interviews make it possible to check for potential additional elements of BI systems' successes or failures (Driscoll et al. 2007). We conducted 15 interviews as part of the qualitative study, and three group interviews with a total of seven participants were arranged for the three organisations. The results from the survey were presented to the groups, and the participants in the group interviews commented on the survey results. Afterwards, we formulated a semi-structured interview. The semi-structured interviews had an average length of 45 minutes.
All interviews were transcribed and analysed in NVivo version 11.0. We used a deductive method to categorise the different tasks' descriptions. The different categories were adapted from earlier work. We will use the interviews to exemplify quantitative findings in the analysis.

RESULTS
The results of the study are presented in three sections: end users' characteristics, task characteristics and the users' assessments of BI success.

End user characteristics
In the survey, the respondents were asked about their gender, age (Table 1), education, organisational role and experience.  Most of the respondents were women (73%). As shown in Table 1, the majority of respondents were 40-49 years old.
Their educational levels varied; most commonly, the respondents either held a master's degree (35%) or a vocational degree (30%) (see Figure 1).
The respondents' organisational roles were distributed among employees (72%), managers (21%) and students (6%) (see Table 2). For the sake of comparison, Negash & Gray (2008) found that BI is mainly used by managers and highly educated employees.
Finally, we can characterise the respondents in terms of their BI experience. The distribution appears in Figure 2. The question asked the respondents to assess their BI experience on a scale from 1 to 5, where 1 is 'little experience' and 5 is 'great experience'.
More than three-fourths (76%) of the respondents rated their BI experience at 3 or below, indicating that they were not highly experienced users. Here, it should be noted that two of the three organisations under investigation had used BI for a number of years, while one implemented BI about two years ago (in 2016). This difference of time spent with BI may explain some of the differences in experience assessments among the organisations. In related studies, technology experience has been found to be a critical factor for system success (Dishaw & Strong, 2003;Marshall et al. 2000;Thompson et al. 1994). Corresponding explanations were found in the interviews. One interview participant explained: "…the more experience you get with the system, the more you think: 'Well, this is fine and really easy to understand'. But then when you get out and have to explain it -for instance at meetings in our controller group, if I have prepared something and ask 'What do you think about this?', then they are like, 'We don't understand that', and I think, 'Well, that is easy to understand'. But you easily get into an understanding of what you think is easy to understandable" (2017).
Apart from confirming the importance of experience, the quote above also illustrates the difference between system users (users interacting with the system) and information users (employees using the information from BI).

Characteristics of BI tasks
We identified BI tasks from several different dimensions in the study. At an overall level, the respondents were asked what BI was primarily used for. The distribution of their answers is shown in Figure 3. More than half (56%) reported their main use is for data extraction, 29.8% point to reporting and the last 14.2% mentioned ad-hoc analysis as their most frequent use of the system. The respondents were also asked what specific BI functionalities they use. The results appear in Figure 4. As shown, the most-used function by far was data filtering, followed by compiling data in a table and visualisation. Less common functionalities included drilling down, layout formatting, calculations (e.g., numeration) and merging (e.g., linking data together from different sources). Ad hoc-analysis Data extraction Reporting Figure 3 What is BI primarily used for? Figure 4 Functionalities used in BI Figure 5 The amount of work tasks in which BI is used Figure 5 presents the share of tasks that BI represents among the total tasks handled by the respondents. As is evident from the figure, the majority use BI less than half of the time, or not at all. Thus, BI use represents a minor part of the total number of tasks respondents handled. However, based on interview data, it appears that, despite these minor use patterns, users still consider BI to be an important tool in their everyday work practice.
All statements in the survey were rated on a 5-point scale, with 1 being highly disagree and 5 being highly agree. As shown in Table 3, the mean response rating is above 3, indicating that the respondents more or less agree with the statement.
The statement with the lowest rating addresses the amount of data in the system and the relation with the respondents' tasks. In the interviews, more participants claimed that they think the amount of data is appropriate. One comment may explain some of the lower rating of the statement. The participant states: "As regards the report module, I can create the things I would like to, but it is less appropriate in terms of publication and dissemination. It comes in short in terms of saying 'We would like to continue here, but we can't with this tool, so we need new technology to move on'" (2017).  Error! Reference source not found. shows the distribution of responses as regards 'Task significance'. In all statements, except for the last one, the mean value is between 3 and 4. In general, the respondents consider their BI tasks to be important, and they or others make decisions on these tasks. The respondents do not consider the tasks to be important for collaboration outside of their organisation, so the BI is instead used as an internal tool. The following quotes from the interviews illustrate the significance of BI tasks: "The tasks are pivotal, because we need to touch upon the economy so much. We need to file reports very, very much" (2018).
"I think it is quite important, at least in relation to many of the requests we get. We get a lot of requests that are used politically, or […] So, if we didn't have the option […]. Usually, it is with a very short time frame, where a politician asks, 'We need this for our…' Or it can be on the same day that you are in a meeting and they go, 'We need this…' and that would then be within an hour" (2017). The quotes illustrate why BI is important to the users. One thing is that they need to file reports within their organisations. The other is that several users receive requests from others regarding facts that are being drawn from the BI system. Task interdependence reflects the users' dependencies in relation to the system; this can be in terms of a user's dependence on something, or another's dependence on the user. These assessments are presented in Table 5. Here, we can see that, across all three organisations, the users do not depend on anything to use BI themselves. It is assumed that the BI system is available, updated and so on. Further, the respondents disagree that their tasks depend on colleagues' completion of other tasks first. Although the ratings for task interdependence are low, the interviews revealed dependencies, typically in the participants' ability to deliver information to other employees, such as managers. To illustrate: "Well, the closest managers" (2017).
"The department management, and then our doctors. They are the ones using me for this" (2017).
"That would typically be our political committees or the management of our administration" (2017).
A partial explanation for the dependencies reported of the respondents may be found in this quote from the interviews: "Everyone can go in and get data. It is just not everyone [who] know[s] how to use it. The benefit of asking me is that I know data better than most people, and by that I also know how to use data and how to do this. That's how it works" (2017).
In sum, one of the barriers to employees' access to the BI system is a lack of knowledge of the underlying data models.
The respondents were also asked about the difficulty of the tasks solved by the BI system. The assessments appear in Error! Reference source not found.. The assessments of the two first statements in Table 6 signal a neutral attitude. The latter statement, concerning the novelty of the tasks, demonstrates that the users, to some extent, consider BI tasks to be routine. Despite their ratings of the statements, the interviews reveal nuances of task difficulty. Thus, the interviews demonstrate examples of both routine and more complex tasks. To illustrate routine tasks, consider: "Well, if I have to do a monthly follow-up, then I need to define and follow up on every cost centre and see the transactions, if they are okay. That is like a routine task" (2017).
"That is when I make a list of the patients we had for the last five years with a specific diagnosis. Super easy task, because the template was developed for that purpose. Some BI people have been thinking big thoughts, and there are very good headings for what you should go and look for in the system, so it is just a matter of going in and typing your filters" (2017).
However, the interviews also reveal examples of more complex tasks. For instance: "We had some where we should combine the kind of medicine they got, which is a standard extract in BI, with how long they were hospitalised. So, they should have had both a certain kind of medicine and be hospitalised for more than five days, for instance" (2017).
Here, the complexity consists of combining different data types. Another kind of complexity is when the underlying data models are complex. For example: "Yes, you need to know your data and which… You could believe that you have the right data and then there is really something you didn't take into account. I think I have tried that quite often, at least in the first couple of years I was working with this. That you think that you had everything under consideration and then there is some kind of twist of it" (2017).
The last cluster of statements, regarding the users' tasks, concerns the specificity of these tasks. The assessments appear in Table 7. The respondents' assessments are average when rating to what extent the tasks are defined before they start solving them. There is a general agreement that the tasks can be solved in different ways. Again, the table indicates some extent of the routine tasks in the low rating of the repeatability of the tasks in the last statement.

Users' assessments of BI success
In addition to the respondents' background characteristics and the characteristics of their BI tasks, the survey also considered system and information quality as independent variables that influence the success of the BI system. The assessments of system quality appear in Table 8. In that table, all statements have mean ratings below 3, meaning that the users find the system difficult to learn, use and understand. The challenges are expressed in the interviews: "It requires quite a lot to learn how to use BI" (2017).
"I would say that, about using the front end part of it, if you haven't used it a lot, then it can be quite difficult to find out how to present it" (2017). When the users have difficulties using the BI system, they report two strategies for the appropriation of the technology (Dourish, 2003). One is asking a colleague for help, which is considered an example of the employee aiming to adopt the technology. The other exemplifies adaptation. Here, the users import the data into Excel: "At times, I import it into Excel. I might as well admit it: I love Excel, including the graphical part. I like working with that" (2017). Information quality is another aspect that influences the users' assessment of BI success. Three statements are included in the construct. The assessments appear in Table 9. Overall, the users have a neutral assessment of the three statements with a mean slightly above 3.
Thus, the users believe the consistency of the data to be reasonable. The data validity is rated slightly higher, while the users' impressions of other employees' impressions receive the lowest, but also most neutral, assessment. Regarding users' satisfaction with the BI system (see Table 10), they do not think that the system's functions and capabilities are as expected (rated at a mean of 2.82). However, they would still recommend the system to colleagues (rated at a mean of 3.21). The overall rating of satisfaction has a mean of 3.07. The interviews revealed some of the issues the users experience with the system. In some cases, the users prefer to report in Excel. For instance: "It is not like it is working in the same way as a spread sheet with formulas and the like. It is a little more complicated and heavy to work with" (2017).
The users' individual impact is lower, when asked if they can make reports in BI effectively (mean of 2.98) and quickly (mean of 2.73). Completing the reports in BI is rated higher (mean of 3.04), suggesting that, although it may not be effective or fast, the users do finish their reports in the system (see Table 11).

DISCUSSION
The data analysis has shown that the majority of respondents and active users of BI are employees, and not managers as found in other studies. To most respondents, BI was not playing a dominant role in their work life, which may also explain their assessment of their own experience as being limited. However, the users handle routine tasks and more difficult tasks in the system. The most important task handled in the BI system was data extraction and more specifically filtering data and merging them into tables. The most important use of BI is internally in the organisations. The users do not think it is very easy to learn how to use the system, but they do experience consistency and validity of the data in the system, and they would recommend it to colleagues. The results of the study can be used to indicate how implementation can be approached to take into account the strengths and challenges users experience in using BI as a part of their work practice. The results demonstrate that the users still can experience challenges in using the system, although the system has been implemented for some time in all three case organisations.
This paper used DeLone & McLean (1992) for guiding the data collection. That enables comparison across diverse organisations for a general picture of BI use and users in the public domain. However, if the aim is a more detailed understanding of the BI tasks and related use in subdomains within this domain, more task-oriented theories as presented in the theory section could generate a more detailed understanding of task characteristics and the system use generated on that basis.

CONCLUSION
This paper has provided a picture of the characteristics of BI users and their tasks carried out by means of a BI system in different parts of the public sector. The analysis has not investigated differences between domains, but merely presents a cross sector perspective. Future research should aim to investigate further the differences between sub domains, the types of tasks generated, and the system success found as a consequence. This could provide useful inputs for the implementation of BI systems in the public sector.