48 lines
1.9 KiB
Markdown
48 lines
1.9 KiB
Markdown
|
|
# Date Format Handling for ClickHouse Events API
|
|
|
|
## Problem Description
|
|
|
|
The event tracking API was experiencing issues with date format compatibility when inserting records into the ClickHouse database. ClickHouse has specific requirements for datetime formats, particularly for its `DateTime64` type fields, which weren't being properly addressed in the original implementation.
|
|
|
|
## Root Cause
|
|
|
|
- JavaScript's default date serialization (`toISOString()`) produces formats like `2023-08-24T12:34:56.789Z`, which include `T` as a separator and `Z` as the UTC timezone indicator
|
|
- ClickHouse prefers datetime values in the format `YYYY-MM-DD HH:MM:SS.SSS` for seamless parsing
|
|
- The mismatch between these formats was causing insertion errors in the database
|
|
|
|
## Solution Implemented
|
|
|
|
We created a `formatDateTime` utility function that properly formats JavaScript Date objects for ClickHouse compatibility:
|
|
|
|
```typescript
|
|
const formatDateTime = (date: Date) => {
|
|
return date.toISOString().replace('T', ' ').replace('Z', '');
|
|
};
|
|
```
|
|
|
|
This function:
|
|
1. Takes a JavaScript Date object as input
|
|
2. Converts it to ISO format string
|
|
3. Replaces the 'T' separator with a space
|
|
4. Removes the trailing 'Z' UTC indicator
|
|
|
|
The solution was applied to all date fields in the event payload:
|
|
- `event_time`
|
|
- `link_created_at`
|
|
- `link_expires_at`
|
|
|
|
## Additional Improvements
|
|
|
|
- We standardized date handling by using a consistent `currentTime` variable
|
|
- Added type checking for JSON fields to ensure proper serialization
|
|
- Improved error handling for date parsing failures
|
|
|
|
## Best Practices for ClickHouse Date Handling
|
|
|
|
1. Always format dates as `YYYY-MM-DD HH:MM:SS.SSS` when inserting into ClickHouse
|
|
2. Use consistent date handling utilities across your application
|
|
3. Consider timezone handling explicitly when needed
|
|
4. For query parameters, use ClickHouse's `parseDateTimeBestEffort` function when possible
|
|
5. Test with various date formats and edge cases to ensure robustness
|