Software Engineer, Big Data Job Description
Software engineer, big data
provides data management expertise to our clients in evaluating requirements and developing data architecture and Big Data solution design.
Software Engineer, Big Data Duties & Responsibilities
To write an effective software engineer, big data job description, begin by listing detailed duties, responsibilities and expectations. We have included software engineer, big data job description templates that you can modify and use.
Sample responsibilities for this position include:
Experience with Java servlet containers or application servers such as JBoss, Tomcast, Glassfish, WebLogic, or Jetty
Distributed (HBase or Cassandra or equivalent) or NoSQL
Research, experiment, and utilize leading Big Data methodologies, such as Hadoop, Spark, Redshift, Netezza, SAP HANA, and Alteryx
Work from a high-level architecture, lead the implementation of an integrated data processing workflow, linking open source technologies with internally development components
Design and develop applications in Java/ Scala using a variety of frameworks and tools
Work with the Project Manager and Development Manager on a day to day basis
Ability to think declarative rather than procedural computer logic
Understanding of relational database system design
Understanding of ETL concepts in manipulation of data
Unix/Linux scripts for Batch scripts
Software Engineer, Big Data Qualifications
Qualifications for a job description may include education, certification, and experience.
Licensing or Certifications for Software Engineer, Big Data
List any licenses or certifications required by the position: GCP, COMPTIA, ITIL, AWS
Education for Software Engineer, Big Data
Typically a job would require a certain level of education.
Employers hiring for the software engineer, big data job most commonly would prefer for their future employee to have a relevant degree such as Bachelor's and Master's Degree in Computer Science, Computer Engineering, Engineering, Technical, Programming Languages, Software Engineering, Education, Mathematics, Science, Information Technology
Skills for Software Engineer, Big Data
Desired skills for software engineer, big data include:
Application
Data and infrastructure architecture disciplines
Architecture
Spark
Algorithms
Design and business processes
SQL
Cassandra
Data structures
Enacting service level agreements and the appropriate escalation and communication plans to maintain them
Desired experience for software engineer, big data includes:
*ECL (Enterprise Control Language) is the powerful programming language that is ideally suited for the manipulation of Big Data
Provide expertise and leadership on making technical decisions and delivering a platform that provides business value
Be innovative and solve complex problems using a variety of tools and technologies
A commitment to quality and operational excellence in addition to speed of delivery
Design, build and deploy, scalable and reliable data pipelines to move large amounts of data to MPP columnar Data Warehouses
Develop solutions to Big Data problems utilizing common tools found in the Hadoop ecosystem
Software Engineer, Big Data Examples
1
Software Engineer, Big Data Job Description
Job Description Example
Our growing company is looking for a software engineer, big data. We appreciate you taking the time to review the list of qualifications and to apply for the position. If you don’t fill all of the qualifications, you may still be considered depending on your level of experience.
Responsibilities for software engineer, big data
- Review architecture and designs by CRM development team to ensure solutions are viable, scalable, and will meet performance standards and requirements
- Contribute to development of architecture documents, user manuals, management reports and other technical documentation
- You’ll scale out the platform to process thousands of events per minute, and be the expert in our team for running our internal systems for stream processing systems and other batch compute engines like Spark and Elastic MapReduce
- Understanding and/or experience with serialization frameworks such as Thrift, Avro, Google Protocol Buffers, and Kyro preferred
- Visualization tools &
- Ensuring the quality & accuracy of the platform, code & data is always on par with the requirements
- Candidate should take full ownership of his deliverables & improve quality over time
- Should be ready to work on any area involving new features or improvement in existing components
- Design, implement and maintain the Crash Error Reporting (CER) backend system
- Design, implement and maintain the ADP Data receiving server layer
Qualifications for software engineer, big data
- Masters Degree in Computer Science or similar degree
- Responsible for designing, developing, and testing software in a multi-platform environment using cutting-edge technology
- Work with system administrators, security analysts, product management and clients to ensure we produce high quality products that exceed expectations
- Experience utilizing NoSQL technologies such as Hbase
- At least bachelor degree in computer science
- Excellent large-scale design, implementation, and testing (C#, C/++, Java, or equivalent)
2
Software Engineer, Big Data Job Description
Job Description Example
Our company is hiring for a software engineer, big data. We appreciate you taking the time to review the list of qualifications and to apply for the position. If you don’t fill all of the qualifications, you may still be considered depending on your level of experience.
Responsibilities for software engineer, big data
- Love working in a creative dynamic environment, with other developers and partners in an agile environment
- Partnering with technology and business teams to understand, model and architect business processes including high availability requirements, time to market, performance characteristics and technical solutions
- Provide consulting and technical expertise to business partners, architects and developers
- Responsible for setting up and driving architectural design and code reviews
- Play an integral role in building and enhancing the infrastructure required to support the CRM capability in terms of robustness, scalability, availability and efficiency
- Works with business architects to ensure development supports the business and IT strategic plan
- Provide technical insight on current platforms to improve performance, availability and utilization optimizations
- Develop, build, test, and deploy applications on-premises or on the cloud using iterative and agile-like development processes
- Design, plan, develop, deploy and maintain working software in a fast-paced environment using agile methods and iterative delivery of business value
- Use and exploit the latest development technologies and frameworks to produce and deploy software
Qualifications for software engineer, big data
- Familiarity with big data system and data analytics
- Knowledge in server side development, familiarity with technologies like Mesos, Docker, AWS, etc
- Must have excellent time, task, and resource organization skills due to the multiple platforms, systems and issues involved
- Expert in Java, Python, Scala or other scripting languages
- Ability to debug, profile and optimize code (memory/speed) to maximize the capabilities of the services
- Program in a methodical and structured manner
3
Software Engineer, Big Data Job Description
Job Description Example
Our company is searching for experienced candidates for the position of software engineer, big data. If you are looking for an exciting place to work, please take a look at the list of qualifications below.
Responsibilities for software engineer, big data
- Design and development of a Big Data platform
- Understanding and enhancing the open source library of Big Data Technology stack
- Development of custom open source library for Big Data platform
- Implementation and Integration of Federated SSO (Single Sign On)
- Development of smart connector to integrate Big Data platform with different data sources
- Process structure and unstructured data
- Real-time Batch processing of big data via various data source including social media
- Collaborate with Technical and Business team
- Integrate solution with various system via Web Service/Message Broker and third party connectors/adaptors
- Define strategy for Pre Processing large dataset and Data transformation
Qualifications for software engineer, big data
- Have experience on multi-thread programming, socket programming and JNI programming
- Familiar with SCM tools include GIT, JIRA
- Bachelor or Master, degree in Computer Science or similar technical
- Passionate about engineering and delivering the best solution for our customers
- High bar for operational excellence
- Desire to learn about and aid in adoption of new technologies and best practices
4
Software Engineer, Big Data Job Description
Job Description Example
Our growing company is looking to fill the role of software engineer, big data. To join our growing team, please review the list of responsibilities and qualifications.
Responsibilities for software engineer, big data
- Contribute to producing technical documentation such as data architecture, data modeling, data dictionary, source to target mapping with transformation rules, data flow design, and test cases
- Work with partners, product managers and developers to develop and review designs and business requirement
- Create new and improved techniques and/or solutions for data collection, management, and usage
- Apply statistical concepts and techniques to analyze product quality, experiments and user behavior
- Support internal team requests for data, work with teams to analyze data and bring value to help the organization take decisions
- Collaborate with partners and drive analytic projects end to end
- Communicate complex quantitative analysis in a clear, precise, and actionable manner to non-technical audiences
- Present the insights you find in a concise and effective manner
- Analyze data covering a wide range of information from logs and historical data feeds from connected vehicles and smart devices
- Communicate complex concepts and the results of the analyses in a clear and effective manner to the team
Qualifications for software engineer, big data
- Excellent coding skill in Java/Python/Ruby
- B.S./BE/Btech or MS/Mtech in Computer Science or related field or equivalent work experience is required
- Knowledge of Big data(Hadoop, Hbase ..)
- Experience with R / Shiny, MLlib, SciPy, NumPy
- Hands on experience with structured unstructured data
- Bachelor degree or an advanced degree (Masters and/or PhD) in computer science, mathematics, data mining, statistics, machine learning, or related field
5
Software Engineer, Big Data Job Description
Job Description Example
Our company is growing rapidly and is hiring for a software engineer, big data. Thank you in advance for taking a look at the list of responsibilities and qualifications. We look forward to reviewing your resume.
Responsibilities for software engineer, big data
- Develops proficiency in the application and use of systems, tools, and processes within the department’s scope
- You like to move fast and get code into production because you know that your work has a positive impact on the end users
- You automate everything you touch
- You optimize the system periodically to provide the best performance
- Provides technical expertise to peers across the company
- Lead technical design and review sessions
- Ensure solutions are scalable and resilient
- Establish strong working relationships with team members and across the business
- Optimize queries and batch/real-time data pipes to improve the performance of data delivery
- Takes up QE tasks of Big Data team tracks and completes the necessary testing ownership
Qualifications for software engineer, big data
- Experience with Java or C++ and knowledge of a scripting language (Perl, Ruby, Python)
- 3+ years of software development experience with Java, Python, Ruby, or C#/C++
- Thorough understanding of scalable, secure and robust n-tier architectures for achieving end-to-end performance, scalability, availability, reliability, usability, configurability, integration and security
- Hands-on ability to build quick prototype/visuals to assist with high-level product concepts and epic stories
- 2 - 3 years of industry experience with Big Data Technologies
- Solid Experience with Hadoop Eco System (Zookeeper, Sqoop, Pig, Hive, MapReduce and Yarn)