Edinburgh/Remote within UK
Up to £49,000 p/a
An innovative and expanding digital solutions agency are looking to add a Data Engineer to their growing Data Analytics team.
The team harness User Research, Data Engineering, Data Science and Data Visualisation to produce open source code, research and tools that help charities, businesses and governments to make better informed choices for the benefit of society.
Working in collaboration with data scientists, designers and behavioural scientists, practitioners at the frontline, academics; and people with lived experience, the Data Analytics team seek to use Big Data for good.
- Developing new strategies for data collection and processing including:
- Scalable machine learning methods
- Sophisticated web-scraping methods
- Data pipelines
- Database development
- Big data technologies
- Ongoing development of new features, whilst supporting the Data Analytics and Production Systems:
- Implementation of new features which broaden our capabilities in Big Data research
- Implementing patches and hotfixes to unstable data pipelines
- Managing database and server loading
- Supporting the management of cloud services:
- Implementing and administrating cloud technologies
Breadth of knowledge is key within this role, so whilst you might not have in depth knowledge of all the technologies listed, a working knowledge in some is more than sufficient.
- At least three year’s professional experience programming in Python (essential)
- Experience developing in a UNIX/LINUX environment (essential)
- Pipeline/CI/CD technologies (Docker, Luigi, Metaflow, Travis etc.)
- Cloud development and/or management using AWS or equivalent cloud technology (EC2, ECS, batch, S3, Lambda, Terraform)
- Database technologies (MySQL, Elasticsearch, Neo4j, Postgres, SqlAlchemy, Alembic)
- Git + GitHub (or equivalent) for version control, whilst working in a collaborative development or production environment
- Text processing pipelines, or machine learning and NLP methods
- Online data collection
- ETL with standard python tools
- REST API development (preferably with Django or Flask)
- Test Driven Development
This is a fantastic opportunity to join a growing data team working to produce Big Data products that change the world for the better.
Working on exciting projects with major clients and on innovating internal systems, the data engineer would take up an instrumental role as part of a compact, dynamic and forward-thinking data team.
Shortlisting next week so apply now.