Date of Employment: June 2016 - Present (6 years 6 months)
Type of Employment: FTE
Title: Site Reliability Engineer / Application Infrastructure Engineer
Date of Employment: February 2012 - March 2016 (4 years 2 months)
Type of Employment: Contractor (first 2 years) and FTE (last 2+ years)
Title: System Engineer
- 2+ years of experience working directly with deploying IPv6 in a large scale Linux environment.
- 4+ years of experience as a founding member of the IT Chef team.
- Built IT Infra's multi-regional Chef infrastructure stack starting with the original implementation.
- Lead engineer writing, testing, and implementing of the IT Chef code base.
- Cookbook creation included basic system configurations to complex system automation workflows.
- Orchestrated 600+ Chef code pushes involving cookbooks, roles, environments, and databags.
- Authored Chef code to manage Chef itself to include managing its chef-clients on Chef nodes and auto-deploying Chef servers with Chef.
- Authored an original password randomnizer in pure Ruby and integrated it with Chef seamlessly to securely randomize all root user passwords.
- Authored a specialized piece of Chef automation code that dynamically dual-stacked RHEL and Ubuntu nodes with IPv6 network interfaces.
- Authored all code to convert Chef cookbook based secrets to use encrypted data bags and migrated all Chef Linux nodes to support encrypted data bags.
- Fully implemented a deployment of taste-tester, a Facebook specific Chef integration testing tool, on the IT Chef stack.
- Self taught the Ruby Programming language both as a declarative DSL within Chef and as a general OOP higher language.
- 4+ years of experience managing thousands of heterogeneous IT Linux systems (CentOS, RHEL, OEL, and Ubuntu) with Chef automation.
- 4+ years of hardening Chef managed Linux systems with things including: LDAP auth, DUO 2-factor auth, OpenSSH security tuning, and automated sudo policies.
- 4+ years of experience building and operating the multi-regional package management infrastructure for different Linux distributions.
- Originally designed, built and implemented a multi-regional Linux driven HTTP heterogeneous package management service using Chef.
- Authored an original package repository synchronization tool in pure Ruby that automatically mirrored public repos to keep them up-to-date.
- Maintained and built all custom IT Infra packages that included RPM and DEB package artifacts.
- Created a lightweight BASH package build script framework to assist in compiling source code based packages.
- Mastered the usage of FPM, a ruby meta-packaging tool, that helped build custom code based packages.
- Worked closely with Security teams to coordinate, push and mitigate packages affected by major security vulnerabilities.
- Ensured all and any package spec files or custom build scripts were version controlled within git repositories.
- 2 years of experience building, re-designing, re-building, and operating multi-regional global Facebook IRC service.
- Rebuilt and migrated the entire Facebook IRC Infra to the IT Infra stack, including different backend systems, load balancers, and networking.
- Designed and implemented the re-designed IRC infrastructure entirely in and with Chef so that it was completely codified infrastructure.
- Configured the IRC implementation to be a multi-faceted Internet facing service.
- Iterated upon the IRC infrastructure by upgrading the whole stack to work with IPv6 making it the first Internet facing IPv6 IT service.
- Coordinated and communicated IRC migrations, outages, and upgrades to the company.
- Managed monitoring IRC service health, users, and traffic by failing regional traffic over during incidents and upgrades.
- 4+ years of experience in a fast paced, iterative development and operations environment.
- 3+ years of experience operating and managing load balancer traffic infrastructure for IT Infra services, focusing in Chef, Package Management and IRC services.
- 4+ years of experience provisioning virtualized Linux machines.
- 4+ years of experience with on-call rotations and on-demand repair of large scale service outages.
- 4+ years of experience with site/service event and incident response, incident root cause analysis, incident report creation, and incident review post mortem meetings.
- 4+ years of experience working in line with Technical Project Managers to push and deliver innovative new services.
- 4+ years of experience working with internal and external auditors to ensure compliance of system configurations for financial based infrastructure.
- 4+ years of experience rebuilding entire infrastructure stacks to migrate and upgrade them to different datacenters locations.
- 2+ years of experience actively participating in Disaster Recovery drills on core infrastructure services to test their resiliency and availability.
- Participated in complete Infrastructure Destruction and Security Breach simulations. Also, know as Red Team exercises.
Date of Employment: September 2010 - January 2012 (1 year 5 months)
Type of Employment: FTE
Title: Unix Systems Administrator
- Managed the engineering Ubuntu Linux development systems with BASH scripting, git code pushes, security package updates, and network monitoring.
- Converted the Slide Design Team to corporate Google hardware and software.
- Implemented a multiplatform SSHFS software deployment for Slide Teams to securely access remote file systems.
- Converted all Slide Team members to use SSH key based authentication when accessing a bastion hosts.
- Created a scripted account creation automation and management process for newly hired, exited, and existing Slide AU employees.
- Improved Slide AU account setup with multiple BASH scripts for LDAP auditing, developmental playground initialization and conversion, file encryption, and Slide AU specific OS configurations installations.
- Authored procedural documentation on Slide AU user account management, developer playground creation, SSH usage and ethics guide, new hires, terminations.
- Assisted the Slide AU Systems Operations team with a major datacenter colocation migration and setup.
- Supported multiple remote teams in China and Europe.
Date of Employment: July 2009 - August 2010 (1 year 2 months)
Type of Employment: Contractor/FTE
Title: Systems Operations Admin and Facilities Assistant
- Worked on the Systems Operations team with an emphasis on Linux systems administration.
- Maintained, operated, and repaired corporate Ubuntu Linux servers and services.
- Deployed FOSWiki, FreeGhost/FOG PXE imaging system, and Apple Netboot System Image Server to improve desktop and laptop distribution.
- Administered the Mailman mailing lists, TWiki, and Kwok Information internal servers and services.
- Supported 100+ local employees and multiple overseas offices using mixed operating systems environments.
- Worked with Slide Facilities to help maintain, organize, and improve the three company office building floors.
Date of Employment: July 2007 - September 2007 (3 months)
Type of Employment: Internship
Title: System Operations Intern
- Collaborated with Systems Operations team members to support all Slide employee desktop and laptop computers.
- Assisted in the physical hardware and software configuration maintenance of corporate Linux servers.
- Collaborated with network engineering to fix and setup new corporate network infrastructure.
- Implemented a new company wiki with TWiki to help improve employee communication, productivity, and sharing of project critical information.
Cal Poly Pomona University Library
Date of Employment: October 2005 - June 2007 (1 year 9 months)
Type of Employment: On-campus Student
Title: Student Systems Operations Admin
- Supported all library employees from six internal departments.
- Effectively maintained and ensured the functional and operational integrity of all computer labs consisting of 100+ computers of mixed operating systems.
- Completed tasks to include resolving support tickets in an efficient manner.
- Solved daily on-demand desktop software and user support problems for library employees.
Last update: 2023-01-16