Software Developers at IBM are the backbone of our strategic initiatives to design, code, test, and provide industry-leading solutions that make the world run today - planes and trains take off on time, bank transactions complete in the blink of an eye and the world remains safe because of the work our software developers do. Whether you are working on projects internally or for a client, software development is critical to the success of IBM and our clients worldwide. At IBM, you will use the latest software development tools, techniques and approaches and work with leading minds in the industry to build solutions you can be proud of.
Your Role and Responsibilities
As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the design of data solutions using Hadoop based technologies along with Hadoop, Azure, HDInsights for Cloudera based Data Late using Scala Programming.
Real time anomaly detection solutions that proactively identify service impacting incidents and prevent system downtimes. This is done by leveraging an ensemble of Deep learning and LSTM models.
Natural Language Processing for entity, topic clusters and relationship extraction
Text Analytics in human generated tickets and correlation with event tickets for event noise reduction. ApplyNatural Language Classification and RNN algorithms to automatically route tickets
Log Analysis - Text mining, message clustering / templatization, Logs to metrics, anomaly detection, event annotation and sequencing
Learn Log Message Sequence for each mainframe batch job and Identify Anomalies during job runs using sequence mining techniques and provide early warning / alerts
Cloud Migration - Patterns-based discovery optimization: Identify potential business application boundaries using algorithmic approach from Cloudscape data.
Wave planner: Employ goal-based reasoning from AI planning capabilities for Server affinity, cost, time, black-out windows, etc.
To power the above use cases, we have a Big Data system that can handle 2-3 TB of data daily and we manage a data lake that is 15 PB in size.
Required Technical and Professional Expertise
12 Years of IT Industry Experience and 8 years relevant exp in data engineering
Working knowledge of microservices using Docker and Kubernetes
Prior experience in document search stores like Elasticsearch and Timeseries DB like InfluxDB
Prior experience with open source data management tools like logstash, FluentD, Nifi, Airflow, Arrow, Flink, Druid, etc
Experience working in one of the Hyperscaler / cloud provider data platform (AWS, GCP Azure, IBM)
Familiarity with IT data and domain is desired, this includes Observability, AIOps, SRE, ITIL, Monitoring, etc
Familiarity with prevalent message bus and queuing technologies (Kafka, RabbitMQ)
A strong background and exposure to databases, including both relational (PostgreSQL, MySQL) and NoSQL (Redis, Cassandra, MongoDB) database systems
Familiarity with realtime streaming and processing of various data sources, including logs, time series telemetry data, unstructured social data and relational data
Preferred Technical and Professional Expertise
Experience working in IT Service delivery and Infrastructure domain,
Knowledge of ITIL Framework
Experience working with large data sets leveraging distributed systems e.g. Spark/Hadoop.
Working knowledge of Relational (Oracle, Db2, SQL, etc) and new age databases (NoSQL databases) e.g. MongoDB, Cassandra
People with demonstrated past projects executed based on above technologies
About Business Unit
At Global Technology Services (GTS), we help our clients envision the future by offering end-to-end IT and technology support services, supported by an unmatched global delivery network. It's a unique blend of bold new ideas and client-first thinking. If you can restlessly reinvent yourself and solve problems in new ways, work on both technology and business projects, and ask, "What else is possible?" GTS is the place for you!
Your Life @ IBM
What matters to you when you’re looking for your next career challenge?
Maybe you want to get involved in work that really changes the world? What about somewhere with incredible and diverse career and development opportunities – where you can truly discover your passion? Are you looking for a culture of openness, collaboration and trust – where everyone has a voice? What about all of these? If so, then IBM could be your next career challenge. Join us, not to do something better, but to attempt things you never thought possible.
Impact. Inclusion. Infinite Experiences. Do your best work ever.
IBM’s greatest invention is the IBMer. We believe that progress is made through progressive thinking, progressive leadership, progressive policy and progressive action. IBMers believe that the application of intelligence, reason and science can improve business, society and the human condition. Restlessly reinventing since 1911, we are the largest technology and consulting employer in the world, with more than 380,000 IBMers serving clients in 170 countries.
For additional information about location requirements, please discuss with the recruiter following submission of your application.
Being You @ IBM
IBM is committed to creating a diverse environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.