Skip to content
Pillow Documentation

Connectors

Connectors are Pillow’s data collection agents that gather real estate information from various sources and submit it to the Mill API.

Overview

Section titled “Overview”

Harvesters provide:

  • Automated data collection from various real estate sources
  • Property data normalization and enrichment
  • Scheduled updates and batch processing
  • Error handling and retry mechanisms
  • Data quality validation
  • JWT-based authentication with Mill API

Key Features

Section titled “Key Features”
  • Multi-Source Support: Connect to various real estate data providers
  • Flexible Scheduling: Configure collection frequency and timing
  • Data Validation: Built-in quality checks and validation rules
  • Fault Tolerance: Automatic retry and error recovery
  • Batch Submission: Efficient bulk property submission to Mill API
  • Authentication: Secure JWT-based authentication

Available Harvesters

Section titled “Available Harvesters”

Current harvesters include:

  • homes-co-nz: New Zealand property listings from homes.co.nz
  • realestate-com-au: Australian property listings from realestate.com.au

Getting Started

Section titled “Getting Started”

Run an existing harvester:

Terminal window
# Navigate to a harvester directory
cd harvesters/sources/homes-co-nz


# Set up environment variables
cp .env.example .env
# Edit .env with your Mill API URL and token


# Run the harvester
export MILL_API_URL="http://localhost:4000"
export MILL_API_KEY="your-token-here"
go run main.go

Creating a New Harvester

Section titled “Creating a New Harvester”

See the harvesters README for detailed instructions on creating new harvesters.

Next Steps

Section titled “Next Steps”