Hadoop Administrator Job Description
Hadoop Administrator Duties & Responsibilities
To write an effective hadoop administrator job description, begin by listing detailed duties, responsibilities and expectations. We have included hadoop administrator job description templates that you can modify and use.
Sample responsibilities for this position include:
Hadoop Administrator Qualifications
Qualifications for a job description may include education, certification, and experience.
Licensing or Certifications for Hadoop Administrator
List any licenses or certifications required by the position: SSL, CE, AWS, SAP, HDP, DOD, TLS/SSL, GCP, II, IAT
Education for Hadoop Administrator
Typically a job would require a certain level of education.
Employers hiring for the hadoop administrator job most commonly would prefer for their future employee to have a relevant degree such as Bachelor's and Master's Degree in Computer Science, Technical, Engineering, Information Technology, Education, Information Systems, Business, Computer Engineering, Administration, Software Engineering
Skills for Hadoop Administrator
Desired skills for hadoop administrator include:
Desired experience for hadoop administrator includes:
Hadoop Administrator Examples
Hadoop Administrator Job Description
- Accountable for storage, performance tuning and volume management of Hadoop clusters and MapReduce routines
- Involves designing, capacity arrangement, cluster set up, performance fine-tuning, monitoring, structure planning, scaling and administration
- Monitor Hadoop cluster connectivity and performance
- SQL and performance tuning at all levels
- Document new environments and develop standards for organization to support a robust
- Directs and participates with team members in the analysis, development and delivery of all support and project work for assigned function(s)
- Capacity planning and implementation of new/upgraded hardware and software releases for storage infrastructure
- Application support (Leavely, Platfora, Securonix, Splunk/Hunk, Katana)
- Change management (svcnow, hardware upgrades/replacements, vendor engagement)
- Support and reengineer Python scripts (Konflux)
- Ability to communicate technical ideas and concepts to less technical audiences
- Experience with industry-standard operational practices such as change management, incident management
- Expert scripting with shell scripting (Bash, PHP, PERL, Python)
- Experience with RDBMS technologies and SQL language
- Experience with Hadoop Ecosystem Security (Kerberos, AD integration, and/or other ecosystem components such as Ranger, Knox, Rhino, Falcon)
- Experience with Amazon AWS and AWS technologies such as EMR, EC2, IAM, S3, Lambda, Data Pipeline, are a strong plus
Hadoop Administrator Job Description
- Frameflow alert creation and active monitoring
- Assets Inventory (Vuln remediation, IAM, AIM, CIITAM, SvcNow, OpsMetrics, internal tracking)
- Security inventory (IIQ groups, service accounts, passwords in EPV, CyberArk cred provider)
- Manage and coordinate platform-wide backups and restores
- Isilon & Cisco Administration
- ESX and VMware administration (configure/deploy VM resources using AXP approved tools)
- Puppet and kick start programming
- Ability to effectively communicate with enterprise storage, network, and backup teams
- Configure and deploy vSphere 5.5, vSphere 6.X, vCenter 5.5, vCenter 6.X, virtual machines, vNetworking (preferred with distributed vSwitches)
- Support systems that conform to architectural guidelines and QA policies and procedures
- This position will be responsible for implementation and support of the Enterprise Hadoop environment, including
- The administrator consultant will have experience in installing, administering, and supporting Windows and Linux operating systems /hardware in an enterprise environment
- Expertise in system administration and programming skills
- Ability to work independently in a team environment on moderate to highly complex issues
- AWS and Google Compute experience a plus
- Five years data mining experience
Hadoop Administrator Job Description
- Working with data application teams and customers to setup new Hadoop users, Linux users, Kerberos principals and LDAP integration and verification of HDFS, Hive, Pig and MapReduce access for the new users
- Performance tuning and troubleshooting of Hadoop clusters and Hadoop MapReduce routines
- Identify, remediate, and track technical issues in a highly complex and dynamic environment
- Plan, coordinate and implement security measures to safeguard information in data warehouse against accidental or unauthorized damage, modification or disclosure
- Learn and execute change/incident management processes and procedures
- Recognize and work appropriately with confidential and sensitive information
- Integrate innovative solutions involving different technologies/products and support the development of testing and acceptance of prototypes and product recommendations
- Create and follow documented processes and procedures while implementing improvements
- Leverage system admin skills for automation, optimization, and operational efficiencies
- Ensure issues are identified, tracked, documented, and reported on and resolved in a timely manner
- 4+ years of administrator experience on Big Data Hadoop Ecosystem and related technologies (Sqoop, Hive, Pig, Flume)
- Prior experience with performance tuning, capacity planning, and workload mapping a plus
- Coding and developing in Hadoop
- 3+ years of strong Linux and Hadoop administration background in a production support setting is required
- Expert in SQL and/or Shell scripting, stored procedures, and construction of highly complex queries is required
- Hands on experience working with unstructured data is required
Hadoop Administrator Job Description
- Have technical knowledge about the administration and control of the Hadoop System, including the associated operating system, related tools, network and hardware
- Creating and maintaining engineering documents and system designs
- Maintain appropriate written documentation for operational procedures and system design
- Experience with monitoring systems and diagnostic tools
- Development, adherence, and enforcement of security standards and policies
- On call production support (24x7)
- Basic understanding of Computer processing operations and Operating Systems
- Review computer systems logs and highlight potential issues for further research
- Write basic computer batch and shell scripts to solve reporting or automation issues
- Add, remove and modify user access to systems with basic uidance and assistance
- Understanding of architecture and support around Big Data and BI solutions is required
- Experience with database and query performance tuning, sizing, and optimization is required
- Deep knowledge of server hardware and configuration is required
- Strong understanding of high availability, fault tolerance, load balancing, and clustering is preferred
- Expert in scripting, stored procedures, and construction of highly complex queries is preferred
- Production support experience with SQL Server, Greenplum, PostgreSQL, or Oracle is a plus
Hadoop Administrator Job Description
- Install basic software licenses for renewals
- Reset file permissions and make modifications to directory structures as needed
- Assist in monitoring and tuning the application
- Remidate system vulnerabilities based on documentation and research
- Seeking college degree in Computer Science or other related field
- December 2017 Graduates
- Exposure to Hadoop//Linux
- Basic understanding of server computing
- Supports the Database Administration functions of the system
- Stewardship the overall Cloudera architecture and infrastructure
- Working knowledge of clustering, replication, and log shipping techniques is a plus
- Support experience and/or knowledge of DASD, Isilon, X15, Kafka, Filemon, Konflux, or Splunk is a plus
- Experience administering Pivotal Hadoop or Hortonworks distribution is a plus
- Experience administering TCP/IP based networks is a plus
- Experience administering & supporting Cisco UCS is a plus
- Experience with performance engineering for storage is preferred