Associate Director / Lead Engineer, GenAI

Job description

The Associate Director / Lead Engineer leads a small team of software engineers responsible for designing and delivering the data pipelines, retrieval systems, and knowledge integrations that power a generative AI platform. This role combines technical leadership with people management — setting architectural direction, removing blockers, and ensuring the team delivers high-quality, scalable solutions. The Associate Director /Lead Engineer balances hands-on technical involvement with strategic planning, stakeholder communication, and team development. Essential Job Functions: Directly manage team(s) focused on delivering high quality project results within one or more major technology disciplines: strategy, software development, operations, engineering, development services, information security, and compliance. Focus on team leadership, delivery of business value, risk mitigation, cost optimization and return on investment. Deliver results based upon FINRA annual goals, department goals and management requests. Direct and coordinate organization's financial and budget activities to fund operations, maximize investments, and increase efficiency for a project(s). Ensure technology best practices are being implemented and improved within team(s). Analyze information and evaluate results to choose the best solutions and solve problems. Support the evaluation of new technologies, techniques, and tools. Report status and issues to senior Technology management team. Work directly with outside vendors to negotiate services and product agreements Willingness to lead, take responsibility and offer options. Serve as a backup to more senior level management as needed. Develop constructive and cooperative working relationships with peers both within and outside of Technology, and maintain them over time. Establish and maintain external relationships with strategic product/service vendors and industry forums (government & private councils, standards bodies, product forums, etc.), as appropriate. Contribute to the establishment, evolution and continued compliance with standard practices and processes within the disciplines. Assist with adherence to technology policies and comply with all security controls. Ensure all work products meets/exceeds FINRA standards and risks are effectively managed. Participate in periodic Disaster Recovery (DR), Business Continuity Planning (BCP) and Sarbanes Oxley (SOX) testing and reporting. Resource Management: Identify and hire resource/skills needed within their organization. Responsible for staff performance management and training. Coordinate assignment of subordinate staff. Demonstration of FINRA’s values. Collaboration, both in-person and virtually, in furtherance of FINRA’s mission of investor protection and market integrity. Key Responsibilities: Technical Leadership & Architecture · Set the technical vision and architectural direction for knowledge framework integrations, including data ingestion, embedding pipelines, and retrieval systems · Make key design decisions on pipeline architecture, data modeling, and search strategies, balancing quality, performance, and delivery timelines · Evaluate and recommend technologies, frameworks, and approaches for knowledge integration challenges · Maintain sufficient hands-on involvement to provide credible technical guidance and unblock the team on complex problems · Demonstrate solid understanding of systems and processes within the domain and recommend improvements Team Leadership & People Management · Manage, mentor, and develop a small team of software engineers, fostering a culture of ownership, collaboration, and continuous improvement · Conduct regular one-on-ones, provide constructive feedback, and support career growth for direct reports · Plan and prioritize team workload, balancing feature delivery, technical debt, and operational responsibilities · Recruit, onboard, and ramp new team members as the team scales · Champion collaborative resolution of defects and lead internal process improvement initiatives Engineering Excellence & Quality · Establish and enforce engineering standards for code quality, testing, documentation, and peer review across the team · Guide the team in best practices for data pipeline development, retrieval system design, and test automation · Ensure comprehensive test coverage including unit testing, integration testing, and retrieval quality evaluation · Drive adoption of test-driven development (TDD) practices and automated quality gates Planning & Delivery · Own the team's roadmap and delivery commitments, translating business requirements into actionable technical plans · Break down complex integration initiatives into well-scoped work items with clear acceptance criteria · Track progress, identify risks early, and adjust plans to meet deadlines without sacrificing quality · Provide regular status updates and delivery forecasts to leadership Stakeholder Collaboration · Partner with product, platform engineering, and agent development teams to align knowledge integration priorities with broader platform goals · Communicate complex or technical information at the appropriate level for diverse audiences including leadership, peers, and external partners · Represent the team in cross-functional planning sessions, architecture reviews, and prioritization discussions · Build strong working relationships across the organization to facilitate collaboration and shared accountability Generative AI & Emerging Technologies · Stay informed of advances in RAG architectures, embedding models, retrieval optimization, and knowledge management approaches · Evaluate emerging technologies and techniques for applicability to the team's work, making build-vs-buy recommendations · Drive innovation within the team by creating space for experimentation and proof-of-concept work Security & Compliance · Ensure all team deliverables meet or exceed organizational standards for secure development practices · O