- Blog
- CV generator
- Community
- Documentation
Fabio Valonga
Sao Paulo, Brazil
I love learning new skills and being updated about the latest technologies. - My top languages are Python/Pyspark, NodeJS/JavaScript, Ruby and I have experience with ReactJS and React Native. - I have FullStack and quality assurance experience, currently working as Data Engineer in a Big Data project My career started as a product manager intern and since then I've gone through several areas such as quality assurance, mobile development, full-stack, and DevOps.
CodersRank Score
What is this?
This represents your current experience. It calculates by analyzing your connected repositories. By measuring your skills by your code, we are creating the ranking, so you can know how good are you comparing to another developers and what you have to improve to be better
Information on how to increase score and ranking details you can find in this blog post.
1,033.6
CodersRank Rank
Top 1%
Associate
Developer
Ruby
Mid
Developer
JavaScript
Mid
Developer
Python
Highest experience points: 0 points,
0 activities in the last year
JavaScript
182.1
exp.
Python
318.9
exp.
Ruby
26.4
exp.
JSON
211.8
exp.
CSS
19.4
exp.
HTML
13.2
exp.
Jupyter Notebook
80
exp.
Shell
17.6
exp.
Java
29.5
exp.
Scala
59
exp.
List your work history, including any contracts or internships
Softensity
Apr 2022 - May 2023 (1 year 1 month)
Data Engineer
Summary:
• Migrating data from RDS to S3 using EMR (Spark/Hadoop)
• Serving data from S3 through REST API (Node Express)
• Creating Data Pipeline using deltalake lib and Kafka
Day-to-day responsibilities:
• Processed historical data creating a pyspark pipeline in Databricks. Coalesce thousands of tiny files into a Delta Table using batch and stream process (Databricks Autoloader) and merge with on-going pipeline.
• Applied business rules to the data using pyspark data wrangling
• Supported the ongoing pipeline adding new datasets using Scala, to read Kafka topics, apply deduplication and business rules then save to S3
• Served data from Rest API using NodeJS + Express
• Defined code patterns for reusable code
Improvements/Accomplishments:
• Created the Historical pipeline responsible to ingest more than 1Pb of data
• Created reports for data quality to help identifying bad/miss data and rules changes
• Supported Monitors jobs to cover pipeline’s health
• Reduced computer costs optimizing clusters usage/resources and applying clean code.
Technology Stack:
• Blockchain and Rest API collectors using Java and Python
• Databricks pipelines using pyspark, Delta Lake and autoloader
• EMR pipelines using spark and scala
• RDS with PostgreSQL Database
• Serve data through REST API on ECS + NodeJS
• Serve data through WebSocket's with Python
• Migrating data from RDS to S3 using EMR (Spark/Hadoop)
• Serving data from S3 through REST API (Node Express)
• Creating Data Pipeline using deltalake lib and Kafka
Day-to-day responsibilities:
• Processed historical data creating a pyspark pipeline in Databricks. Coalesce thousands of tiny files into a Delta Table using batch and stream process (Databricks Autoloader) and merge with on-going pipeline.
• Applied business rules to the data using pyspark data wrangling
• Supported the ongoing pipeline adding new datasets using Scala, to read Kafka topics, apply deduplication and business rules then save to S3
• Served data from Rest API using NodeJS + Express
• Defined code patterns for reusable code
Improvements/Accomplishments:
• Created the Historical pipeline responsible to ingest more than 1Pb of data
• Created reports for data quality to help identifying bad/miss data and rules changes
• Supported Monitors jobs to cover pipeline’s health
• Reduced computer costs optimizing clusters usage/resources and applying clean code.
Technology Stack:
• Blockchain and Rest API collectors using Java and Python
• Databricks pipelines using pyspark, Delta Lake and autoloader
• EMR pipelines using spark and scala
• RDS with PostgreSQL Database
• Serve data through REST API on ECS + NodeJS
• Serve data through WebSocket's with Python
Hvar Consulting Services
Dec 2021 - May 2023 (1 year 5 months)
Tech Lead - Data Engineer
(Part-time job)
Summary:
• Migrate data from SAP DW to AWS Redshift for PowerBI reports.
Day-to-day responsibilities:
• Architecture design for data pipelines
• Help the squad with technical blockers
• Mentored squad members and accompanied them in their career
• Defined code patterns for reusable code
Improvements/Accomplishments:
• Created a generic framework that supports all data wrangling.
• Refactored the framework to run in AWS Glue, AWS EMR, and Azure Databricks
• Reduced computer costs by optimizing cluster usage/resources and applying Spark Structured Streaming
Technology Stack:
• AWS Glue, Redshift, Athena, AWS S3
• Azure Databricks
Summary:
• Migrate data from SAP DW to AWS Redshift for PowerBI reports.
Day-to-day responsibilities:
• Architecture design for data pipelines
• Help the squad with technical blockers
• Mentored squad members and accompanied them in their career
• Defined code patterns for reusable code
Improvements/Accomplishments:
• Created a generic framework that supports all data wrangling.
• Refactored the framework to run in AWS Glue, AWS EMR, and Azure Databricks
• Reduced computer costs by optimizing cluster usage/resources and applying Spark Structured Streaming
Technology Stack:
• AWS Glue, Redshift, Athena, AWS S3
• Azure Databricks
Dextra
2 years 10 months
Data Engineer
Sep 2020 - Apr 2022 (1 year 7 months)
Summary:
• Create NLP pipelines using Airflow as ETL/orchestrator + Google Cloud Dataproc/Azure Databricks and save data into datalake using pyspark + DeltaLake.
• Support TTS + Dataproc pipelines.
• Monitoring performance + issues with sentry.io + ganglia
Day-to-day responsibilities:
• Mentored interns and accompany them in their career plan
• Mentored some employees in the data engineer career
• Maintained Speech to Text pipelines
• Enhance pipelines monitors
• Ensure best code practices through CI/CD validations and GitFlow
• Created NLP pipelines using Airflow as ETL/orchestrator + Google Cloud
• Dataproc/Azure Databricks and save data into data lake using pyspark + Delta Lake.
• Remove sensitive data using Regex and hash functions
Improvements/Accomplishments:
• Migrated all pipelines from Databricks do Google Cloud Dataproc
• Created a single framework that could be reused to scale up new projects and the current projects
• Implemented CI/CD pipelines to avoid code errors and keep production environment healthy
• Implemented usage of performance + issues monitor with sentry.io + ganglia
• Reduced Big Query costs using Presto/Trino
• Migrated pHp application from customer’s server to Google Cloud and made this reliable.
• Created domain-model to standardize inputs making code reusage possible.
Technology Stack:
• Google Cloud Platform + Azure Databricks, DataProc, Compute Engine
• Firebase tools, Google Domain, Google Load Balancer
• Seq2Seq models
• TensorFlow models, MLFlow, scikit-learn, pyTorch
• Create NLP pipelines using Airflow as ETL/orchestrator + Google Cloud Dataproc/Azure Databricks and save data into datalake using pyspark + DeltaLake.
• Support TTS + Dataproc pipelines.
• Monitoring performance + issues with sentry.io + ganglia
Day-to-day responsibilities:
• Mentored interns and accompany them in their career plan
• Mentored some employees in the data engineer career
• Maintained Speech to Text pipelines
• Enhance pipelines monitors
• Ensure best code practices through CI/CD validations and GitFlow
• Created NLP pipelines using Airflow as ETL/orchestrator + Google Cloud
• Dataproc/Azure Databricks and save data into data lake using pyspark + Delta Lake.
• Remove sensitive data using Regex and hash functions
Improvements/Accomplishments:
• Migrated all pipelines from Databricks do Google Cloud Dataproc
• Created a single framework that could be reused to scale up new projects and the current projects
• Implemented CI/CD pipelines to avoid code errors and keep production environment healthy
• Implemented usage of performance + issues monitor with sentry.io + ganglia
• Reduced Big Query costs using Presto/Trino
• Migrated pHp application from customer’s server to Google Cloud and made this reliable.
• Created domain-model to standardize inputs making code reusage possible.
Technology Stack:
• Google Cloud Platform + Azure Databricks, DataProc, Compute Engine
• Firebase tools, Google Domain, Google Load Balancer
• Seq2Seq models
• TensorFlow models, MLFlow, scikit-learn, pyTorch
Backend Engineer + React Native Developer
Jun 2019 - Sep 2020 (1 year 3 months)
Summary:
• Create and support microservices for a telecommunication environment.
• Create and support react-native modules/screens for the main app (Android/iOS).
Day-to-day responsibilities:
• Created and supported microservices for a telecommunication environment.
• Created and support react-native modules/screens for the main app Android/iOS.
• Ensure the code quality for all contractors during code reviews
Improvements/Accomplishments:
• Created monitor dashboards on Kibana and Dynatrace to help identify server problems
• Implemented a new microservice to integrate with Zendesk and reduce throughput to keep the same quota usage
• Implemented fallbacks rules to keep servers reliable during pandemic.
• Implemented rules to detect mobile credit recharge frauds.
Technology Stack:
• AWS environment, Kubernetes, ECS, Jenkins, Docker
• NodeJS + KoaJS
• ReactNativeJS
• Kibana, Dynatrace
• Create and support microservices for a telecommunication environment.
• Create and support react-native modules/screens for the main app (Android/iOS).
Day-to-day responsibilities:
• Created and supported microservices for a telecommunication environment.
• Created and support react-native modules/screens for the main app Android/iOS.
• Ensure the code quality for all contractors during code reviews
Improvements/Accomplishments:
• Created monitor dashboards on Kibana and Dynatrace to help identify server problems
• Implemented a new microservice to integrate with Zendesk and reduce throughput to keep the same quota usage
• Implemented fallbacks rules to keep servers reliable during pandemic.
• Implemented rules to detect mobile credit recharge frauds.
Technology Stack:
• AWS environment, Kubernetes, ECS, Jenkins, Docker
• NodeJS + KoaJS
• ReactNativeJS
• Kibana, Dynatrace
Add some compelling projects here to demonstrate your experience
AutoIT Gem
AutoIT is a ruby gem similar to Selenium except for this control anything made with Windows Frames as Calculator.
This section lets you add any degrees or diplomas you have earned.
Instituto Federal de Educação, Ciência e Tecnologia de São Paulo - IFSP
Bachelors's degree in System Development, Information Technology
Jan 2013 - Jan 2016
IFSP is a Federal Institution that operates within the CTI, The Renato Archer Information Technology Center (CTI) is a research unit of the Ministry of Science, Technology, and Innovation (MCTI) that operates in research and development in information technology. The intense interaction with the academic sectors, through several partnerships in research, and industrial, in several cooperation projects with companies, keeps CTI in the state of the art in its main areas of action, such as the area of electronic components, microelectronics, systems, displays, software and IT applications such as robotics, decision support software and 3D technologies for industry and medicine.
ETEP - Escola Técnica de Paulínia
Certificate in Chemistry, Chemistry
Jan 2010 - Jan 2012
In Brazil, students can get Certificate Program to work in a specific field before ingressing into a university.
This Certificate allows me to work in Laboratory for Chemistry experiments related (QA, research, ...).
This Certificate allows me to work in Laboratory for Chemistry experiments related (QA, research, ...).
Fine Tune BERT for Text Classification with TensorFlow
Jul 2021
Natural Language Processing with Probabilistic Models
Jul 2021
Natural Language Processing with Classification and Vector Spaces
Jun 2021