Manager, Data Engineering Job Description
Manager, data engineering
provides architecture and technology leadership across Batch and Streaming data processing platform in Big Data and Cloud Data technologies.
Manager, Data Engineering Duties & Responsibilities
To write an effective manager, data engineering job description, begin by listing detailed duties, responsibilities and expectations. We have included manager, data engineering job description templates that you can modify and use.
Sample responsibilities for this position include:
Architect and develop integrated data quality applications from the ground up using a modern technology stack such Spark, Scala, Elastic Search, Angular JS, Kibana, Kafka, NiFi
Work directly with Product Owners and end-users to develop data quality solutions in a highly collaborative and agile environment
Help grow and mentor team members on the fine art of software engineering, data quality, and product evaluation
Be an evangelist at hackathons and other external events for the FinTech industry
Lead and oversee development teams focused on K12’s learning analytics, predictive and recommendation engines, and adaptive curriculum capabilities
Lead and participate in defining technical solutions, architecture improvements, technical reviews, and code reviews
Work cooperatively with development team, Product Management, and Network Operations to ensure a successful, well communicated, high quality release
Develop, mentor, and coach development teams to build a strong development team, increase team effectiveness, and promote individual team members’ growth
Collect operational metrics on the effectiveness of the teams
Conduct performance reviews aligned with the corporate policies and timelines
Manager, Data Engineering Qualifications
Qualifications for a job description may include education, certification, and experience.
Licensing or Certifications for Manager, Data Engineering
List any licenses or certifications required by the position: AWS, PMP, PMI, CPA, EMR, ITIL, CCIE, TDWI, CDCMP, CDCDP
Education for Manager, Data Engineering
Typically a job would require a certain level of education.
Employers hiring for the manager, data engineering job most commonly would prefer for their future employee to have a relevant degree such as Bachelor's and Master's Degree in Computer Science, Engineering, Mathematics, Technical, Statistics, Business, Science, Education, Math, Military
Skills for Manager, Data Engineering
Desired skills for manager, data engineering include:
SQL
SQL and relational database constructs
Numpy
Python libraries like Pandas
Relational databases
Scipy and OOPS Concepts
With experience writing SQL queries and stored procedures
Big Data Architecture Patterns and experience in delivery of end to end Big data solutions either on premise
Cloud and social media technologies and integrations
Data technologies in at least one cloud environment
Desired experience for manager, data engineering includes:
4+ years of experience working with frameworks such as Spark, Hadoop, Hbase
2+ years of experience implementing ELK stack
Familiarity with Agile engineering practices
Create software development LOEs for feature requests and assist with forecasting and roadmaps
Define, improve, and promote software engineering processes and best practices
Lead and collaborate with other Engineering Managers, Software Architects, and Technical Leads in defining and maintaining platform architecture
Manager, Data Engineering Examples
1
Manager, Data Engineering Job Description
Job Description Example
Our company is growing rapidly and is looking for a manager, data engineering. If you are looking for an exciting place to work, please take a look at the list of qualifications below.
Responsibilities for manager, data engineering
- Directly supervise 10 - 15 Full-time Equivalent (FTE) regular employees and/or contractors
- Management of assigned staff development (performance management, merit, promotions, ) and policy compliance, all aspects of staff hiring, resource planning, & assignments
- Be responsible for the group’s technical strategy and roadmap – creating success metrics in close collaboration with internal customers
- Work with tech leads and senior engineers to gather system requirements, review new feature proposals and large system improvements
- People management and communication are key skills for this role
- Responsible for leading a team of data engineers who populate and maintain the Data Foundry for CPA
- Leading engineering team in design, development and rollout of Big Data systems and services
- Partnering with system architects, functional managers and program managers to deliver to market mission critical data systems
- Developing new and enhance existing features for collection of massively distributed data systems owned by DDE group
- Working on data collection, processing and access subsystems
Qualifications for manager, data engineering
- 5+ years experience with NoSQL implementation (Mongo, Cassandra, a plus)
- Familiarity with other quality standards and methodologies
- Manage and develop a team of Data and BI Engineers
- Knowledge and direct experience using business intelligence reporting/visualization tools (OBIEE, Business Objects, Cognos, Tableau, MicroStrategy, SSAS Cubes)
- Staff management (performance management, merit, promotions, hiring, training skill development, ) experience
- BS/MS degree, or foreign equivalent, in Computer Science, Business Information Systems, Applied Mathematics, Engineering, or a closely related discipline
2
Manager, Data Engineering Job Description
Job Description Example
Our company is looking to fill the role of manager, data engineering. If you are looking for an exciting place to work, please take a look at the list of qualifications below.
Responsibilities for manager, data engineering
- Working on projects that focus on system scalability, performance, and security
- Following SW development methodology best practices
- Inspiring and enable innovation, help team members to achieve their full potential
- Lead and develop sustainable data driven solutions with current new gen data technologies to meet the needs of our organization and business customers - Designs robust systems with an eye on the long term maintenance and support of the application
- Handle multiple functions / roles for the projects / Agile teams - Has an intrinsic knowledge of standards and languages
- Leverage and establish standards across the team and organization
- Design end-to-end engineering solutions for business opportunities using existing on premises or new Cloud based Big Data technology platforms
- You will be involved in building high-quality, scalable data streaming solutions using Kafka, Kenesis, Flume, Spark, Storm
- As an engineer you will be working on Apache Kafka platform that powers millions of events a day
- Deploy & manage solutions based on services offered by AWS & third party suppliers
Qualifications for manager, data engineering
- TDWI certification or ScrumMaster certification is a plus
- BA/BS or higher in Computer Science (or equivalent work experience)
- Technical capacity to partner with tech leads and be comfortable diving into the fray to help drive resolution of incidents or technical disagreements
- Prior experience in the big data space, ideally as an individual contributor working on open source technologies such as Hadoop, Scalding, Spark, Pig, or Storm
- Previously managed a team of 4-8 engineers with a responsibility for building big data systems
- 8+ years of experience in software development and in that 3+ years of experience as technical manager or a technical lead
3
Manager, Data Engineering Job Description
Job Description Example
Our company is hiring for a manager, data engineering. Thank you in advance for taking a look at the list of responsibilities and qualifications. We look forward to reviewing your resume.
Responsibilities for manager, data engineering
- Work with the on-team Quality Engineers to build automation and test processes across all projects in development
- Support the recruitment program within Data Engineering across different teams within the group
- Advise Senior Management and other stakeholders on feature development, performance, and scalability of the current work in progress
- Create consensus while working with senior associates to quickly resolve issues and technology questions
- Keep up with Big Data trends and communicate opportunities, solutions and issues from a technology perspective to the Data Engineering group as a whole
- Partner with Devices Central FP&A, online and offline channel finance, product management finance, and operations and development finance to develop a shared reporting platform reporting tools to support custom specialized reporting needs
- Lead engineering teams to develop new features and services
- Guide the development of scalable data architecture
- Build infrastructure and services that have immediate impact on our partners and customers
- Responsible for designing, engineering, developing, and delivering an information technology infrastructure that supports enterprise Data Network infrastructure at PH&S
Qualifications for manager, data engineering
- Extensive IT professional experience
- Very Strong experience with integration technologies
- Strong experience with infrastructure, software, applications and database systems
- 8+ years experience in design and development of distributed systems
- 8+ years experience developing software with C/C++ or Java
- 2+ years experience leading/managing successful software engineering teams
4
Manager, Data Engineering Job Description
Job Description Example
Our company is hiring for a manager, data engineering. Please review the list of responsibilities and qualifications. While this is our ideal list, we will consider candidates that do not necessarily have all of the qualifications, but have sufficient experience and talent.
Responsibilities for manager, data engineering
- Responsible for assisting with the development of the operating and/or capital budgets
- Responsible for the ongoing development and revision of performance standards for all job categories within the team
- Ensure systems are engineered for optimal availability, reliability, scalability, and performance
- Maintain the strict confidentiality of all information, including financial/operational, employee/human resource, healthcare/patient data and information
- Manages all data network engineering functions for the enterprise
- Lead a globally distributed team to deliver high quality solutions
- Ensures that their team objectives are completed on schedule, and within a defined budget, following established and agreed procedures and timelines
- Communication normally relates to projects and departments
- Assigns tasks in alignment with project milestones, and oversees their completion
- Owns management of staff, teams, and matrix resources to meet service delivery SLAs
Qualifications for manager, data engineering
- 5+ years of experience developing big data based applications working with technologies including Hadoop, Storm, Hive, Hbase, splunk, Kafka, Spark, data architecture
- Ability to evaluate the risks and strong decision making skills in a fast moving environment
- Experience with Big Data systems (Hadoop, Spark, Hive, etc) and principles (Map/Reduce, etc)
- Experience with scripting (Python, bash, etc) and RESTful API
- Knowledge of software development and design principles
- At least 2 years combined experience in data technologies including Cassandra, Accumulo, HBase, Spark, Hadoop, HDFS, AVRO, MongoDB, Zookeeper
5
Manager, Data Engineering Job Description
Job Description Example
Our company is growing rapidly and is looking to fill the role of manager, data engineering. Thank you in advance for taking a look at the list of responsibilities and qualifications. We look forward to reviewing your resume.
Responsibilities for manager, data engineering
- Define and track project schedules by coordinating with internal and external teams
- Build & manage diverse team of data engineers building and maintaining an enterprise data warehouse platform with its associated ETL processes and data architecture requirements
- Responsible for designing, building, and maintaining robust ETL & data warehouse environment
- Provide data architecture and data pipeline leadership and guidance
- Evaluate, recommend, and implement proper tools & technology to achieve a high performing data warehouse platform servicing thousands of users and a broad set of use cases
- Build cross functional relationships with data analytic teams and business leaders
- Constantly evaluate and optimize deployment & ETL processes to achieve greater efficiency and reliability
- Work with business and IT leadership to align priorities and initiatives
- Establish strong relationships with various internal and external teams to gain an understanding of their strategies, objectives, and tactics to develop a comprehensive corporate executive dashboards
- Assess specific operational issues/areas (such as what work gets done in what regions, cost benefit analysis, establishing best practices, creating models for business problems, etc) for improvement and develop an approach/design for addressing the same through detailed analysis and/or interaction with key departments/functions
Qualifications for manager, data engineering
- Successful candidates will possess strong, demonstrable skills in Linux, Hadoop, systems automation, and AWS Technologies
- Expect to tell us what you’ve shipped and what’s flopped
- Experience building and maintaining user-facing libraries, API’s is a big plus
- It would be awesome if you have a robust portfolio on GitHub and/or open source contributions of which you are proud to share!
- 3+ years with Distributed Computing
- 3+ years with Hadoop & other Big Data Technologies