Big Data Developer Job Description
Big Data Developer Duties & Responsibilities
To write an effective big data developer job description, begin by listing detailed duties, responsibilities and expectations. We have included big data developer job description templates that you can modify and use.
Sample responsibilities for this position include:
Big Data Developer Qualifications
Qualifications for a job description may include education, certification, and experience.
Licensing or Certifications for Big Data Developer
List any licenses or certifications required by the position: AWS, II, IAT, DB, SDLC, AI, AGILE, IC, SSL
Education for Big Data Developer
Typically a job would require a certain level of education.
Employers hiring for the big data developer job most commonly would prefer for their future employee to have a relevant degree such as Bachelor's and Master's Degree in Computer Science, Engineering, Technical, Information Systems, Science, Information Technology, Mathematics, Graduate, Software Engineering, Education
Skills for Big Data Developer
Desired skills for big data developer include:
Desired experience for big data developer includes:
Big Data Developer Examples
Big Data Developer Job Description
- Defines, develops or modifies application modules or enterprise-wide software systems using disciplined software development processes and makes modules production ready by moving them to libraries, completing forms, following procedures, completing version control documents
- Develop Hadoop and Web-Service based applications and components (50%)
- Interface with Users, BAs, and Developers to walk through data design (30%)
- Define comprehensive unit test coverage of data transformations
- Demonstrating analytical technical skills
- Should have basic knowledge in BSS data model specially CRM, OMS, Billing
- Be part of a team of Hadoop developers working on Big Data project with ample opportunity to grow professionally, functionally and technically
- Work with other groups to ensure smooth delivery with high standards and quality
- Understand requirements, propose and deliver superior solutions
- Write well-structured beautiful clean code
- Understanding of Big Data Hadoop Ecosystem components (Sqoop, Hive, Pig, Flume)
- Experience with Java or Big Data Technologies a plus
- Any big data experience with Hadoop, HDFS, cluster management
- 6 years' experience in designing & developing enterprise application solution for distributed systems
- Experience with Hive, Pig & Map Reduce and Hadoop ECO system Framework
- Experience with HBase, Talend, NoSQL databases-Experience Apache Spark or other streaming big data processing preferred
Big Data Developer Job Description
- Experience with Big Data technologies will be a plus (Hadoop, Spark, Kafka, Hbase, etc)
- Write SQL queries to validate the dashboard output
- Working experience with database environment - understanding relational database structure and hands-on SQL knowledge to extract/manipulate data for variance testing
- Performing code reviews and pair programming
- Supporting and enhancing current applications
- Design, develop, test, and implement the applicationInvestigate and resolve complex issues while supporting existing applications
- You will work under general direction within a clear framework of accountability
- Be part of the growing Hadoop community
- Act as senior developer on big data security analytics technologies
- Rapidly develop prototype solutions, integrating a variety of open-source and proprietary technologies
- Solid understanding of data structures & common algorithms Understanding of time-complexity of algorithms
- Expert-level skills and experience in Java or any distributed programming experience
- Expert-level skills and experience in Hadoop ECO System
- 6 years experience in designin & developing enterprise application solution for distributed systems
- Experienve with Hive, Pig & Map Reduce and Hadoop ECO system Framework
- Knowledge on Python & Scala
Big Data Developer Job Description
- Help drive an agile, iterative, DevOps-based lifecycle
- Contribute novel ideas for extracting security intelligence from data
- Identify and document Data Quality rules
- Design, translate business requirements, develop, test, monitor, troubleshoot and administrate our big data environment using our ETL/Data Virtualization/BI Reporting/Big Data Management toolsets, and participate in an on-call rotation
- Participate in maintenance and upgrades for our Big Data platform and toolsets
- Participate in data governance initiatives, maintain metadata (data definitions, relationships, sources, and consumers), and contribute to the company’s Big Data strategies and roadmaps
- Contribute to the support and resolution of incidents which involve all data platforms which includes on-call rotation
- Communicate and accurately document all architectural decisions, plans, goals, functional requirements, and open items and issues to key stakeholders, including management, development teams, and business areas
- Participates in short- and long-term planning sessions with customers to improve business processes and works closely with Technical Review Board and architects to assure all systems are in line with IT long-term strategy
- Assists in developing buy vs
- Strong knowledge loading from disparate data sets
- Good skills and experience in Linux, Java, XML, JSON, REST
- Experience with Hadoop and other related technologies
- Deep knowledge of Hive, SQL, databases and data warehouses
- Fluency in Unix
- High attention to detail and constructive attitude
Big Data Developer Job Description
- Manages customer's priorities of projects and requests
- Assesses customer needs utilizing a structured requirements process (gathering, analyzing, documenting, and managing changes) to prioritize immediate business needs and advising on options, risks and cost vs
- Design and implementation of software product (BigData related) including data models and visuzlisations
- Demonstrate participation with the teams you work in
- Deliver good solutions against tight timescales
- Be pro-active, suggest new approaches and develop your capabilities
- Share what you are good at while learning from others to improve the team overall
- Show that you have a certain level of understanding for a number of technical skills, attitudes and behaviours
- Delivering great solutions
- Being focused on driving value back into the business
- Pre-processing using Hive / Pig / python/ Unix scripting
- Familiarity with map-reduce and Spark
- A deep understanding of algorithms and data structures
- Experience with an open source software contributor, including GitHub, code.google.com, or apache.com
- Preferred 1 year of production Hadoop experience
- Excellent planning/organizational, problem-solving, analytical, and communication skills, strong presentation and decision-making skills
Big Data Developer Job Description
- Working with the other principals and developers to oversee and drive agile SDLC
- Collaboratively work with development peers through iteration tasks (pair programming, code reviews)
- Create and implement simple, pragmatic designs to solve real problems
- Takes initiative and proactively communicates within and among the team...a real collaborator
- Collaborate with testing team to ensure productive code
- Use MapReduce to process large data sets
- 3+ years of overall software development experience in an object oriented language
- Recruiting, leading and inspiring a team capable of delivering great solutions
- Form part of the leadership team of DACE Development demonstrating the necessary leadership skills and style
- Strive to become a luminary in data development leadership
- Experience developing code on distributed databases using Spark, HDFS, Hive
- 5+ years of experience in Application Developer / Data Architect, or equivalent role
- Strong knowledge of data and data models
- Good understanding of data consumption patterns by business users
- Solid understanding of business processes and structures
- Basic knowledge of the securities trading business and risk