Our Data Lab MSc student Rebecca Clancy has spent her internship working at Scottish Enterprise, using her newly acquired academic knowledge to solve some real life problems, and actively learning new methods, applications, software and other skills essential for life as a data scientist.
Working at Scottish Enterprise has been, in a word, surreal. It has been a whirlwind of experiences all crammed into a twelve-week period; the practical skills we have learned here have been broad, deep and encompass several layers of data science. Scottish Enterprise gave us a rich opportunity in several areas and just like a chameleon changes its colour, we swung between factions of data scientist, consultant and business analyst.
Real life data mining projects – messy and complicated
We gained valuable experience heading two data mining projects, which were messy and complicated when compared to the controlled environment we previously had in our University lectures, but this is real life and real life is never straight forward!
Our first project was analysing historical data from Scottish Enterprise and identifying whether, a combination of services yielded a greater impact on the performance of a company. Then, our second project was dealing with unstructured data and text analytics – which is a hot topic nowadays. The model we developed for the text analytics phase was able to detect the keyword which is chosen by the user and provide a similarity score which produces an upper and lower confidence interval or a 100% match for the key word. Then, the lower confidence level is dropped, while if the upper confidence is detected an email is automatically sent to an end-user to investigate the body of the email, and if it is detected with 100% confidence this triggers an automatic email alert to another user. Both phases were highly beneficial to our professional development.
We felt a trusted part of the team
Now, the reason we describe our experience with Scottish Enterprise as surreal is due to the level of exposure and guidance we had within the company. The trust that the company had in us was inspiring as we were allowed entrance into copious amounts of meetings within several departments to help us gain understanding of how the business runs and how these departments interact with each other. This knowledge was a vital stage in the business understanding process and allowed us to develop a relevant and feasible hypothesis to test during our first data mining project.
Additionally, every team was welcoming and enthusiastic to help in any way they could, we never felt like interns but instantly part of the team. There is a great sense of community within the organisation and a huge emphasis on knowledge sharing – we even got to host some knowledge exchange sessions and we also had the opportunity to hold a Strategic Options Development and Analysis (SODA) workshop! This made the internship highly unique and valuable as we were actively learning, gaining practical experience in the theoretical techniques we learned in University and we were being introduced to new methods, applications and software almost daily.
One aspect I loved was an idea brought forward by our Director, Glenn Exton, he spoke about disruptive thinking – challenging yourself and others to ‘upset the standard’. This fosters a highly productive environment as everyone is always challenged to push harder and do better, which is how innovation happens!
Development of essential data scientist skills
We faced many complications during the data collection stage of our project (very common in the life of a data scientist), which encouraged us to investigate merging techniques in Python to produce a robust spreadsheet to analyse – skills that are paramount when considering a career in data science. He also encouraged us to engage with various staff members to ask for spreadsheets or information that could help us move our project forward – techniques again that are essential in the life of a data scientist.
Furthermore, our director again compelled us to be accountable for each step taken and advised us to document every stage of the process. This provided us the opportunity to practice process improvement methods and get some insights into the necessary steps during a company’s digital transformation phase, like Kotter’s 8 step model for transformation and Govindarajan’s ‘Three Box Solution’ for innovation and change. The first data mining project gave us experience in many aspects of organisational operations and project management. These include but are not limited to:
- crucial need for business understanding
- vital to have feasible project plan and aims
- importance of correct data collection methods
- the need to document a project to be transparent and repeatable
- essential to have iteration and feedback from team to ensure project progression
Disruptive thinking to shake up the status quo
Our second data mining project brought back in the concept of disruptive thinking, Glenn suggested Text Analytics to shake up the status quo yet he gave us complete freedom to develop our own project plan and model.
Our second data mining project allowed us to gain experience in:
- text analytics methods and approaches
- new software like SQL and Microsoft Flow
- the benefits that automation has on productivity and agility
- the need for new ways of working and innovation to be embedded into an organisation
We gained extremely valuable technical experience in text mining, pre-processing and text analysis techniques but also some consultancy experience. We were allowed to showcase our model, explain the business benefits associated with the implementation of the model, describe future improvements of the model and ignite the curiosity of analytical tools to discover insights and support decision making. This was disruptive thinking in action – making people question the standard and long for the art of the possible!
