Job Details:
Data Engineer
Job Summary: Incumbents of position will be responsible for designing, building, and optimizing data pipelines to expand the capabilities and products delivered by our analytics platform. Must be comfortable interfacing and supporting multiple teams and working with product managers, data engineers, data scientists, business analysts, and project managers. The ideal candidate has a passion for data, significant experience building production level data pipelines, and enjoys working in a self-directed semi-structured environment. Incumbents of this position will regularly exercise discretionary and substantial decision-making authority.
Essential Job Duties and Responsibilities:
- Evaluating technology stacks, including data storage and databases
- Database maintenance and performance tuning; query optimizations;
- Analyzing, profiling, documenting data warehouses
- Writing technical documentation and business documents
- Implementing best practices, i.e., using code repositories, documenting procedures, refactoring code to be more robust and componentized
- implementing complex big data projects with a focus on collecting, parsing, managing, analyzing and visualizing large sets of data to turn information into insights using multiple platforms
- Building configurable, parametrized, scalable, robust pipelines/ETL workflows
- Scraping, data ingestion and data integration from multiple sources and databases, including web APIs, Protobuf feeds, SQL databases
- End-to-end pipeline QA
- Deploying applications and libraries to cloud platforms
- Feature extraction and generation
- Using Python to clean data, extract features, and build models using packages such as scikit-learn, numpy, pandas
Education:
Bachelor’s degree in computer science or equivalent
Experience:
- 5+ years of hands-on web data engineering experience
- Advanced working SQL and Python knowledge
- Experience bringing data pipelines into production
- Build process supporting data transformation, data structures, metadata, dependencies and workload management
- Database design, schema definition, and database optimization
- Experience building elastic, scalable APIs
- Experience developing in an Agile environment
- Familiar with cloud computing, Azure and/or AWS, PaaS, IaaS
- Experience developing Python scripts
- Experience using BitBucket or other Git tools for version control
- Experience with Linux, developing shell scripts and basic system administration functions
Knowledge, skills and abilities:
- Strong analytical and problem solving skills, attention to details, critical thinking ability, and creativity.
- Excellent written and verbal communication skills used to effectively communicate and clearly present complex information in a manner appropriate to the audience
- Prior experience working in the transportation or logistics industries is a plus
Please forward your resume in word format to kevin@libertyjobs.com
#associate
#midsenior
#IT