bq load: “BigQuery error in load operation: Not found: Project …”


bq load: “BigQuery error in load operation: Not found: Project …”



I'm trying to load some ndjson data. First, creating a table works flawlessly:


> bq mk --table <project-id>:<my-dataset.newtable> newtable.schema.json
Table '<project-id>:<my-database.newtable>' successfully created.



However, the bq load command does not:


bq load


> bq load --source_format=NEWLINE_DELIMITED_JSON <project-id>:<my-dataset.newtable> gs://<project-id>.appspot.com/newtable.ndjson
BigQuery error in load operation: Not found: Project <project-friendly-name>



Please also note also:



I have no problem running the job from BigQuery's web interface.



I have set <project-id> as the default project through the bq init command, but I get the same error, even when creating a table, when I don't specify it.


<project-id>


bq init



Is there an issue with some environment variables that have not been set correctly ?




2 Answers
2



The bq load command usually follows the following structure.



bq --location=[LOCATION] load --source_format=[FORMAT] [DATASET].[TABLE] [PATH_TO_SOURCE] [SCHEMA]



As in the standard bq load command, you don't have to mention the project if you are loading data within the same project that you have logged in you cli. Also you need to mention the schema unless you have auto detect flag set in you command.



The following command allows you to identify the project that you have access to.



gcloud config list





Note that the schema was defined above, on table creation with bq mk --table [PROJECT_ID]:[DATASET].[TABLE] [SCHEMA], last parameter !
– Wiil
Jul 2 at 0:01


bq mk --table [PROJECT_ID]:[DATASET].[TABLE] [SCHEMA]





And thanks a lots for the gcloud config list command, turned out to be really useful
– Wiil
Jul 2 at 1:01


gcloud config list



Ok. Interestingly and unlike with bq mk, with bq load, selecting the <project-id> with [PROJECT_ID]:[DATASET].[TABLE], or throught bq init (and the --location=[LOCATION] option with a fully-qualified Cloud Storage URI as a file) is still irrelevant.


bq mk


bq load


<project-id>


[PROJECT_ID]:[DATASET].[TABLE]


bq init


--location=[LOCATION]



I still had to run either:


gcloud config set project <project-id>


bq load --project_id=<project-id> ...



or


gcloud init



So to sum up, this works:


bq load --project_id=<project-id> --source_format=NEWLINE_DELIMITED_JSON <my-dataset.newtable> gs://<project-id>.appspot.com/newtable.ndjson






By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

-

Popular posts from this blog

How to add background colour in existing image using Swift?

How to add background colour in existing image using Swift? I am trying to add background colour to existing image but it is not working. Here below is my code:- extension UIImage { func imageWithColor(tintColor: UIColor) -> UIImage { let rect = CGRect(x: 0, y: 0, width: 1, height: 1) UIGraphicsBeginImageContextWithOptions(rect.size, false, 0) tintColor.setFill() UIRectFill(rect) let image: UIImage = UIGraphicsGetImageFromCurrentImageContext()! UIGraphicsEndImageContext() //self.type(of: init)(ciImage: CIImage(image: image)!) return image } } extension UIImage { /** Returns an UIImage with a specified background color. - parameter color: The color of the background */ convenience init(withBackground color: UIColor) { let rect: CGRect = CGRect(x: 0, y: 0, width: 1, height: 1) UIGraphicsBeginImageContext(rect.size); let context:CGContext = UIGraphicsGetCurrentContext()!; context.setFillColor(color.cgColor); context.fill(rec...
Read more

Moria Casán

Image
Moria Casán Casán in 2016 Born Ana María Casanova ( 1946-08-16 ) August 16, 1946 (age 71) Buenos Aires, Argentina Residence Parque Leloir, Ituzaingó, Buenos Aires Province, Argentina [1] Occupation Actress, presenter, theatre producer, TV personality, talent manager Years active 1970–present Spouse(s) Mario Castiglione ( m.  1986; div.  1990) Children Sofía Gala Castiglione Signature Ana María Casanova (born August 16, 1946), known by her stage name Moria Casán , is an Argentine actress and TV personality. Casán made her theatre debut in 1970, and quickly became one of the country's leading vedettes during Argentina's Golden Age of revue. Her sex symbol status was further cemented during the late 1970s and 1980s, as she starred in various sexual comedy films alongside Alberto Olmedo, Jorge Porcel and Susana Giménez. Since the mid-1980s, Casán has also established herself as an important TV personality, hostin...
Read more

How to make file upload 'Required' in Contact Form 7?

How to make file upload 'Required' in Contact Form 7? I have a Contact Form 7 form on my Wordpress website and I have specified that the file upload is mandatory by putting a * , however the users are able to submit the form without file upload. * Relevant Contact Form 7 form is given below: <div class="filelink">[file* AddanImage filetypes:gif|png|jpg|jpeg]</div> How do I write the Contact Form 7 code or add a validation perhaps so that the users are not able to submit the form without uploading an image? Any hints/help/guidance is highly appreciated. Have you check doc - contactform7.com/file-uploading-and-attachment – Mukesh Panchal Jun 30 at 8:38 @MukeshPanchal I have and the code complies with the documentation – Hamza Ahmad Jul 1 at 10:44 ...
Read more