Software Engineer

2 days ago

Apply Now

Receive Emails with Similar Jobs

Anthropic

WebsiteLinkedInAll Job Openings

Anthropic is an AI safety and research company working to build reliable, interpretable, and steerable AI systems.

51 - 200

Description

• Lead the “Tokens: Data Acquisition” team. • Develop and maintain an internet scale web crawler responsible for crawling for accessible internet data. • Build required pipelines to quickly ingest data from potential partners for data quality assessments or from other sources. • Build a set of 1-off deep crawlers to improve recall for data from especially beneficial sources. • Build tools for improving the observability and debuggability of crawler system. • Collaborate with team members on improving data acquisition processes. • Participate in code reviews and debugging sessions.

Requirements

• Believe that advanced AI systems could have a transformative effect on the world, and are interested in helping make sure that transformation goes well. • Are interested in building a large scale system to acquire all openly accessible information on the internet. • Extensive experience with building and running large distributed systems. • Familiarity with the non-technical tradeoffs of internet-scale crawling (data privacy, robots.txt adherence, etc.). • Technical expertise: Quickly understanding systems design tradeoffs, keeping track of rapidly evolving software systems. • Familiarity with Cloud-based compute and storage solutions. • Familiarity with Python.

Benefits

• Optional equity donation matching. • Comprehensive health, dental, and vision insurance for you and all your dependents. • 401(k) plan with 4% matching. • 22 weeks of paid parental leave. • Unlimited PTO – most staff take between 4-6 weeks each year, sometimes more! • Stipends for education, home office improvements, commuting, and wellness. • Fertility benefits via Carrot. • Daily lunches and snacks in our office. • Relocation support for those moving to the Bay Area. • Private health, dental, and vision insurance for UK-based employees. • Pension contribution (matching 4% of your salary). • Health cash plan. • Life insurance and income protection.