Python AI Engineer

Home - Careers - Python AI Engineer Python AI Engineer Home - Careers - Python AI Engineer Python AI Engineer Learn more about us LOCATION: Hanoi, HCMC Job Description We are looking for a Python AI Engineer to join us in building and advancing our AI Indexing & Search Pipeline. In this role, you will be responsible for designing and implementing end-to-end pipelines for data crawling, content processing, vector embedding generation, and indexing to power high-performance semantic search capabilities. Pipeline Development & Core Architecture: Develop, optimize, and maintain the Python-based Batch AI Indexer system. Build and operate an end-to-end data pipeline Build and optimize Semantic Search / Vector Search capabilities. Data Crawling & Processing: Crawl and collect content from diverse web sources (Homepage, Naver Blog, and other relevant web data sources). Optimize text chunking strategies and data preprocessing tailored for language models. AI Models & Search Engine Integration: Utilize OpenAI Embeddings for content vectorization. Integrate and manage Elasticsearch 8.x for vector storage and search (working extensively with 1536-dimensional dense_vector and the Korean Nori Analyzer). Utilize FAISS for local vector search experiments and rapid prototyping. Batch, State & Reliability Management: Design and implement mechanisms for Incremental Indexing and Full Re-indexing. Manage incremental batch states using Redis. Track batch execution history and metadata using PostgreSQL. Implement robust retry mechanisms and error-handling logic to ensure high system reliability. Scheduling, Deployment & Operations: Configure automated job scheduling using APScheduler or Linux Crontab. Deploy and operate batch applications on Linux environments using systemd and Python virtual environments (venv). Perform logging, monitoring, and troubleshooting for batch job operations. Collaborate closely with Backend, AI, and DevOps team members to deploy and scale system operations. Required Skills And Experience Core Experience & Technical Skills: 2–4+ years of strong hands-on development experience in Python. Proficiency in web scraping and crawling libraries (e.g., BeautifulSoup, Scrapy, Playwright, or Selenium). Practical experience with Elasticsearch 8.x (specifically Vector Search, dense_vector, kNN search, and custom analyzer configurations like Nori). Deep understanding and hands-on experience with Embedding models (e.g., OpenAI Embeddings, HuggingFace) and text chunking/preprocessing techniques. Understanding business logic, designing solutions, and leading development Have e-commerce development experience is preferred Databases & System Architecture: Solid skills in Redis (for caching/state management) and PostgreSQL (or equivalent relational databases). Strong knowledge of Batch Processing architectures, state management, idempotency, and error handling within data pipelines. Experience with FAISS or other vector search libraries for local experimentation. DevOps & Environment: Proficient in Linux environments, application deployment via systemd, and Python virtual environments. Experience with scheduling tools (crontab, APScheduler). Strong command of Git, logging frameworks, and system troubleshooting. Nice-to-Have Qualifications: Experience with Korean Natural Language Processing (Korean NLP). Experience with LLM orchestration frameworks such as LangChain or LlamaIndex. Hands-on experience with Docker, Kubernetes, or CI/CD pipelines. Benefits Competitive and negotiable salary (UP TO 50M) Probation salary is 100% of official salary 13th-month salary and performance review twice a year Bonus for special occasions each year (Labor Day, National Day, Solar New year, Lunar New Year) Project bonus Employee’s professional certification and training allowances subject to company regulations Opportunity to participate in internal training programs on AI, foreign languages, and professional development Health Care Insurance Annual Health Assessment Social, health and unemployment insurance following government policy Enjoy company summer trips and other team building activities held monthly and quarterly Professional, creative and dynamic working environment Work five days per week with flexible check-in time CONTACT Send CV to: [email protected] Hotline: +842466666100 (Recruitment Team) Facebook GEM Careers: https://www.facebook.com/tuyendungGEM Linkedin GEM Careers: https://www.linkedin.com/company/gem-careers-hiring/