Databricks Resident Solution Architect

As a Databricks Resident Solution Architect, you will serve as a key technical advisor and subject matter expert for our clients, focusing on the design, implementation, and optimization of Databricks solutions. This position is a senior level customer-facing role that requires deep expertise in Apache Spark™ along with a breadth of big data solution architecture experience. You will work closely with clients to understand their business objectives, technical requirements, and data challenges, and collaborate with cross-functional teams to deliver innovative solutions that leverage the full potential of Databricks platforms. This role requires a combination of technical expertise, client-facing skills, and strategic thinking to drive successful outcomes and customer satisfaction.

Responsibilities:

• Consultation and Needs Assessment: Engage with clients to understand their business goals, data challenges, and technical requirements, and provide expert guidance on leveraging Databricks platforms to address their needs.
• Solution Design: Design end-to-end solutions using Databricks platforms, considering factors such as data ingestion, processing, analytics, machine learning, and scalability to meet client objectives.
• Architect production level workloads, including end to end pipeline load performance testing and optimization
• Implementation and Integration: Lead or support the implementation and integration of Databricks solutions within client environments, collaborating with data engineers, data scientists, and other stakeholders as needed.
• Best Practices and Optimization: Define and promote best practices for Databricks usage, configuration, and optimization, ensuring clients maximize the value and performance of their Databricks deployments. 
• Technical Leadership: Serve as a technical leader and advocate for Databricks solutions, staying informed about the latest features, enhancements, and best practices, and sharing knowledge with clients and internal teams.
• Proposal Support: Collaborate with the sales and solutions teams to provide technical inputs for proposals, including solution architecture, infrastructure requirements, and implementation strategies.
• Pursuits: Engage in discussions with potential clients to understand their technical requirements and objectives and provide guidance on how our Databricks solutions can address their specific needs.
• Demonstrations: Conduct product demonstrations and presentations to showcase the features, functionality, and performance of Databricks platforms, tailored to the requirements and specific use cases of prospective clients.
• Proof of Concept (PoC) Initiatives: Lead or support the execution of PoC initiatives to validate the feasibility and effectiveness of Databricks solutions in addressing client challenges and objectives.
• Client Relationship Management: Build and maintain strong relationships with key client stakeholders, serving as a trusted advisor and point of contact for technical inquiries, escalations, and support.
• Documentation and Knowledge Sharing: Document solution designs, architectures, and implementation details, and share knowledge and insights with clients and internal teams through presentations, training sessions, and written materials.

Qualifications:

• Bachelor's degree or higher in Computer Science, Engineering, or a related field.
• Proven experience as a solution architect, data engineer, or similar role, with expertise in designing and implementing data and analytics solutions.
• In-depth knowledge of Databricks platforms, including model selection, model lifecycle, hyper-parameter tuning, model serving, deep learning and using tools like Apache Spark, Delta Lake, Unity Catalog, MLflow, and associated components.
• Experience with Development Tools for CI/CD, Unit and Integration testing, Automation and Orchestration, REST API, BI tools and SQL Interfaces. E.g. Jenkins.
• 5+ years' Experience working as either: Software Engineer/Data Engineer: query tuning, performance tuning, troubleshooting, and debugging Spark and/or other big data solutions.
• Strong understanding of cloud platforms such as AWS, Azure, or GCP, and experience with related services (e.g., S3, Blob Storage, GCS).
• Understanding of Data governance applications such as Immuta, Collibra etc
• Proficiency in programming languages such as Python, Scala, or SQL.
• Excellent communication and presentation skills, with the ability to convey complex technical concepts to non-technical audiences.
• Strong problem-solving skills and the ability to work effectively in a dynamic and collaborative environment.
• Experience with agile methodologies and project management practices is a plus.
• Certification in Databricks or related technologies is preferred.

If you are passionate about leveraging data and analytics to drive business outcomes and thrive in a client-facing role where you can make a meaningful impact, we encourage you to apply for this exciting opportunity to join our team as a Databricks Resident Solution Architect.