Data is the foundation of effective Twitter automation. Whether you're targeting specific audiences, analyzing competitors, or building engagement lists, BotWitter's Data Fetcher API gives you powerful scraping capabilities.
In this guide, you'll learn how to collect user profiles, scrape tweets, search hashtags, and retrieve follower/following lists using BotWitter's built-in API.
Data Collection Use Cases
Competitor Analysis
Scrape competitor tweets and engagement metrics to understand what content performs best.
Lead Generation
Find users interested in specific topics by searching hashtags and collecting follower lists.
Trend Monitoring
Track hashtags and keywords to spot emerging trends in your niche.
Engagement Targeting
Build lists of active users to target with follows, likes, and replies.
Scraping Tweets
Collect tweets from specific users or get details about individual tweets:
# User tweets by username
GET /api/v1/data-fetcher/user/{username}/tweets-by-username?limit=20
# User tweets by ID
GET /api/v1/data-fetcher/user/{userId}/tweets?limit=20
# Tweets AND replies
GET /api/v1/data-fetcher/user/{userId}/tweets-and-replies?limit=20
# Single tweet details
GET /api/v1/data-fetcher/tweet/{tweetId}
# Quote tweets of a tweet
GET /api/v1/data-fetcher/tweet/{tweetId}/quotes?limit=20
Tweet Data Structure
Content
id- Tweet IDtext- Tweet contentcreated_at- Timestampurl- Tweet URLlang- Languagesource- Posted from
Engagement
like_countretweet_countreply_countquote_countview_countfavorited- You likedretweeted- You RT'd
Metadata
is_retweetis_quoteis_replyhashtags- Arraymentions- Arrayauthor- User object
Search Tweets
Search for tweets by hashtag or text query. Perfect for finding trending content and target users.
Hashtag Search
Search tweets containing a specific hashtag:
Example: /search/tag/crypto
Text Search
Search tweets containing specific keywords:
Example: ?query=AI tools
Search Parameters
| limit | Number of tweets to return (1-100, default 20) |
| product | Latest (recent tweets) or Top (popular tweets) |
product=Latest for fresh content and active users. Use product=Top for high-engagement tweets.
Follower & Following Lists
Retrieve follower and following lists for your accounts. Great for building target lists and analyzing audience.
# Get followers of your account
GET /api/v1/accounts/{id}/followers?limit=100
# Get accounts you're following
GET /api/v1/accounts/{id}/following?limit=100
User Data in Lists
Each user in the list includes:
Timeline Data
Access your account's home timeline and your own tweets:
# Home timeline (tweets from people you follow)
GET /api/v1/accounts/{id}/home-timeline?count=20
# Your own tweets and replies
GET /api/v1/accounts/{id}/tweets-and-replies?limit=20
Tweet Validation
Verify if a tweet exists and is accessible before taking action:
# Validate tweet ID
GET /api/v1/data-fetcher/tweet/{tweetId}/validate
Validation Results
Tweet exists
Bad format
Deleted
Private account
Author banned
Cache System
BotWitter caches scraped data to reduce API calls and improve performance:
View Cache Stats
Returns cache size, duration, and cooldown settings.
Clear Cache
Force fresh data on next request.
API Examples
cURL Examples
# Get user info
curl http://localhost:8080/api/v1/data-fetcher/user/elonmusk/info
# Search #crypto tweets (latest 50)
curl "http://localhost:8080/api/v1/data-fetcher/search/tag/crypto?limit=50&product=Latest"
# Get account followers
curl http://localhost:8080/api/v1/accounts/1/followers?limit=100
Quick Recap
- User Info: Full profile data including followers, bio, verification
- Tweet Scraping: User tweets, single tweet, quotes, tweets+replies
- Search: Hashtag and text search with Latest/Top sorting
- Lists: Followers and following for your accounts
- Timeline: Home feed and your own tweets
- Validation: Check if tweets exist before actions
- Cache: Automatic caching with manual clear option