Big Data Software Engineer Job Description
Big Data Software Engineer Duties & Responsibilities
To write an effective big data software engineer job description, begin by listing detailed duties, responsibilities and expectations. We have included big data software engineer job description templates that you can modify and use.
Sample responsibilities for this position include:
Big Data Software Engineer Qualifications
Qualifications for a job description may include education, certification, and experience.
Licensing or Certifications for Big Data Software Engineer
List any licenses or certifications required by the position: GCP, COMPTIA, ITIL, AWS
Education for Big Data Software Engineer
Typically a job would require a certain level of education.
Employers hiring for the big data software engineer job most commonly would prefer for their future employee to have a relevant degree such as Bachelor's and Master's Degree in Computer Science, Computer Engineering, Engineering, Technical, Programming Languages, Software Engineering, Education, Mathematics, Science, Information Technology
Skills for Big Data Software Engineer
Desired skills for big data software engineer include:
Desired experience for big data software engineer includes:
Big Data Software Engineer Examples
Big Data Software Engineer Job Description
- Contribute to other areas of the ADP as needed
- Analyze requirements, design, write, test and deploy highly functional code
- Actively participate in the team by making recommendations on how to improve product quality group productivity
- Actively participate in team by making recommendations on how to improve product quality group productivity
- Profile, monitor, undertake performance-analysis, optimization, scaling and performance tuning
- Keep up to date on current trends in the field of big data
- Enjoy working with data – data analysis, data quality, reporting, and visualization
- Visualization tools & libraries, reporting tools, Splunk (Hunk), Tableau, d3.js
- Work with cloud and on-premises product teams to understand data flow and locations and security concerns of information providers
- Divide and coordinate tasks with other development engineers
- Knowledge/working experience on different BigData technologies - Hadoop, Map Reduce, YARN, Spark, Hive, Kafka, Avro, Cascading
- Experience with developing cloud services
- Experience in working on applications with either millions of users or processing terabytes of data per day
- At least bachelor degree in computer science or very similar discipline
- Previous experience Paid and Organic Customer Acquisition , Adwords, Facebook Advertising
- Experience with a data warehouse technology (Oracle, Teradata, Netezza, Redshift, HDFS Hive ) and relevant data modeling practices
Big Data Software Engineer Job Description
- Collaborate with with a diverse team of engineers and product managers to improve our machine learning infrastructure and data quality
- Participate in the design, planning and development of solutions based on business requirements
- Participate as part of a project team to design, code, unit test and implement moderately complex programs to support the portfolio management process
- Monitor and report to management on project status, identify issues that may impede attainment of goals, recommend and implement corrective measures
- Review and fix problems uncovered by testing or customer feedback
- In this role, you will be responsible for learning our various systems and be able configure them in a QA environment for testing purposes, and exercise them with a combination of automation technologies
- Taking guidance from senior engineers in the areas of design and implementation and build upon your skill sets accordingly, and applying the knowledge gained for future work
- Identify issues and escalate suggestions for database improvements (tables, indexes, DAL changes)
- Troubleshoot operational DB issues (isolate problem queries or connections, capture and analyze DB traces)
- This individual will apply attained experiences and knowledge in solving complex problems
- Motivated, self-starter, creative thinker, ability to work alone or as part of a team
- Bachelor’s degree from an accredited college or university in Computer Science, Computer Engineering, Engineering, or a related field with five years of experience
- Lead and participate in analyzing requirements, develop system architecture and designs which meet requirements
- Design and create automated unit and functional tests for production code
- Lead, coach, and help develop more junior engineers on the team
- Advise management on development estimates, technical information, and issue resolution, on architecture and technology choice points
Big Data Software Engineer Job Description
- Familiarity with machine learning concepts
- Work on high-quality architecture and implementation design, collaborating with the customer, system/business analysts and onshore architects
- Integration with Notification service via Email, SMS
- You will work on data solutions that support and enable product and business teams at EAN to make data driven decisions
- You will be part of cross-functional team alongside Engineers, Data Scientists and Data Users
- You will collaborate with upstream system architects, developers and program managers to resolve data issues
- Build scalable and high-performant code
- Mentor other less experienced big data engineers within the team
- Work in a devops environment
- Work in an agile and dynamic environment
- Team player with drive to grow others around them as much as themselves
- Experience with big data/advanced analytics concepts and algorithms
- Experience with machine-learning algorithms, libraries that assist in the integration of those algorithms (Spark MLlib, Weka, R, Mahout)
- Familarity with collaborative development environments
- MS or PhD in Computer Science, preferably with a focus on Statistical Learning, Applied Statistics, Marketing Science, Operations Research
- Experience with large systems software design and development, with extensive knowledge of UNIX/Linux
Big Data Software Engineer Job Description
- Be an integral member in the design, analysis, development and deployment efforts for multiple enterprise application projects to solve complex business problems
- Leads the QE efforts for Big Data team and takes complete ownership of releases
- Helps in building automation framework and CI/CD
- Creates test plans, designs, develops, and executes reusable and maintainable automated scripts
- Works with developers to analyse requirements, and reproduce problems to get them resolved
- Familiarity with pub/sub messaging systems such as Google Cloud Pub/Sub or Apache Kafka
- Experience as a team lead or other technical leader
- You will be building core data capabilities and services and be passionate about software engineering and data
- You should care deeply about writing solid code
- You closely follow industry trends and the open source community and an avid learner
- Experience with Storm, Spark, HBase, Cassandra, Kafka, etc a plus
- Ability to work in a fast-changing and dynamic environment - Excellent interpersonal, verbal and written communication skills
- At least 3 years knowledge of statistics and experience using statistical packages for analysing large datasets (Excel, SPSS, SAS, R)
- Bachelors or Advance Degree in Computer Science or like field with 7+ years of experience is required
- Willingness to work in an agile, team development environment
- A programming language
Big Data Software Engineer Job Description
- Works with developers to analyze requirements, and reproduce problems to get them resolved
- Improve the enterprise systems by integrating existing solutions and building new technology
- Implement software which efficiently analyzes very large scale data sets (hundreds of millions of transactions)
- Manage large data sets in regards to insertions, updates and deletions
- Implement intermediate data layers to facilitate fast reporting results
- Implement software which has a low failure rate
- Implement software which can adapt to changes in coming data
- Troubleshoot issue resources
- Implement software with inherent low maintenance overhead
- Develop software that can be monitored and tracked for failures
- Excellent knowledge of designing and implementing NoSQL databases as Cassandra or similar
- Experience on database design on Hadoop/NoSQL
- Security implementation of a Hadoop/NoSQL environment
- Design and build scalable platform involving truly cutting edge technology and massive amounts of data
- Research and analyze application behaviors and improving performance and stability
- Work within our global network to optimize applications for linear scaling