S3 Query Exception (Fetch)

I have uploaded data from Redshift to S3 in Parquet format and created the data catalog in Glue. I have been able to query the table from Athena but when I create the external schema on Redshift and tried to query on the table I'm getting the below error

ERROR: S3 Query Exception (Fetch)
DETAIL:
-----------------------------------------------
error: S3 Query Exception (Fetch)
code: 15001
context: Task failed due to an internal error. File 'https://s3-eu-west-1.amazonaws.com/bucket/folder/partition_key/filename.parquet_1 has an incompatible Parquet schema for column 's3://bucket/folder
query: 560922
location: dory_util.cpp:717
process: query1_118_560922 [pid=32409]
-----------------------------------------------


The queries are workinh well in Athena

1 Answer
1

It kind of tells you what's wrong - the schema of table/partition and the file contents differ too much. The easiest way to fix that would be to run a crawler over the data location with the "update each partition definition from table" checked.

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.