Gtjkyu

XPATH syntax validator in Python I develop a crawler with many actions. Many xpaths are involving and for this reason I use a json file for storing. Then crawler start running I would like to make a basic syntax check (before xpath usage) on xpaths and raise error for invalid xpaths. for example: xpath1 = '//*[@id="react-root"]/section' xpath2 = '//*[[@id="react-root"]/section' xpath3 = '//*[@id="react-root"]section' from these xpaths only xpath1 is valid is there any module or regex which does this kind of validation? 1 Answer 1 You can compile the xpath strings with lxml.etree.XPath which will raise an exception if the syntax is incorrect: lxml.etree.XPath >>> import lxml.etree >>> lxml.etree.XPath('//*[@id="react-root"]/section') //*[@id="react-root"]/section >>> lxml.etree.XPath('//*...

How to zip unequal lists as a product of first list with others? I have multiple lists like following- list1=['Tom'] list2=[16] list3=['Maths','Science','English'] list4=['A','B','C'] I want to zip these lists to achieve the following mapping- desired results- [('Tom', 16, 'Maths','A'), ('Tom', 16, 'Science','B'), ('Tom', 16, 'English','C')] Result i am getting by using the following command- results=zip(list1,list2,list3,list4) [('Tom', 16, 'Maths','A')] this is just an example of my problem.If a generalised solution is provided it would be helpful. If I use the statement- res= itertools.izip_longest(*[x[c] for c in cols]) I am getting multiple rows but getting null for the name and age column. Also consider passing the column names in the above way since the names of columns are not static. Possible duplica...

How to convert csv into mock when csv file is specified as argument in unit test of Django command There is a django command that gives the path of the csv file as an argument as follows. class Command(NoticeCommand): def add_arguments(self, parser): parser.add_argument( '--file', dest="file", type=str, required=True ) def handle(self, *args, **options): with open(options['file'], 'r') as f: reader = csv.reader(f) next(reader) for row in reader: ... I am thinking to make this csv file mock when doing unit test. However, I do not know which part and how to make mock. Also, the argument is required = True . How can I call UnitTest when csv is mocked? required = True from mock import patch from django.core.management import call_command class ImportCsvTest(TestCase): @patch("common.management.commands.import_csv.????") def test_import_csv(self...

Change default expiry period of “Pyotp” I'm using Pyotp for generating OTP. Default expiry of this token is 30sec. How can I change the default expiry to some 60 seconds? This is the simple code I'm using. secret = pyotp.random_base32() totp = pyotp.TOTP(secret) otp = totp.now() 1 Answer 1 Here a try , It was worked for me: totp = pyotp.TOTP(secret, interval=60) By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Django model “doesn't declare an explicit app_label” I'm at wit's end. After a dozen hours of troubleshooting, probably more, I thought I was finally in business, but then I got: Model class django.contrib.contenttypes.models.ContentType doesn't declare an explicit app_label There is SO LITTLE info on this on the web, and no solution out there has resolved my issue. Any advice would be tremendously appreciated. I'm using Python 3.4 and Django 1.10. From my settings.py: INSTALLED_APPS = [ 'DeleteNote.apps.DeletenoteConfig', 'LibrarySync.apps.LibrarysyncConfig', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', ] And my apps.py files look like this: from django.apps import AppConfig class DeletenoteConfig(AppConfig): name = 'DeleteNote' and from django...

is django compressor default on django install? Is django compressor default on django install? I have been having an issue with whitenoise and django. I keep getting this error: ValueError: Missing staticfiles manifest entry for 'inline.bundle.js' ValueError: Missing staticfiles manifest entry for 'inline.bundle.js' now all the research I have done points that it may be a whitenoise (im using heroku) and a issue with django compressor. But I do not remember installing such a tech. Is it by default? One of the work arounds suggested is to replace django compressor with another. But how? whitenoise Actively researching this, if anyone has had this issue before would appreciate a work around. That looks like an Angular issue. And no, django-compressor is not installed by default unless it is installed by a third party library. – Selcuk Jul 2 at 3:38 ...

How can I add files to GitLab with `python-gitlab`? Using Django, I’d like to sync the files in the database with git repositories on my GitLab instance via python-gitlab . python-gitlab Here you can find my Python code: import gitlab import base64 import os from .models import Meme from django.conf import settings class Sync: def sync () : gl = gitlab.Gitlab('<GitLab URL>', private_token='xxxxxxxxxxxxxx') for meme in Meme.objects.all(): meme_title = meme.meme_title meme_file = str(meme.meme_file) root = settings.MEDIA_ROOT place = os.path.join(root, meme_file) # Create a new project on GitLab. project = gl.projects.create({'name': meme_title }) data = { 'branch': 'master', 'commit_message': 'Automatic commit via sync.py.', 'actions': [ { ...

How to collapse multiple columns into one in pandas I have a pandas dataframe filled with users and categories, but multiple columns for those categories. | user | category | val1 | val2 | val3 | | ------ | ------------------| -----| ---- | ---- | | user 1 | c1 | 3 | NA | None | | user 1 | c2 | NA | 4 | None | | user 1 | c3 | NA | NA | 7 | | user 2 | c1 | 5 | NA | None | | user 2 | c2 | NA | 7 | None | | user 2 | c3 | NA | NA | 2 | I want to get it so the values are compressed into a single column. | user | category | value| | ------ | ------------------| -----| | user 1 | c1 | 3 | | user 1 | c2 | 4 | | user 1 | c3 | 7 | | user 2 | c1 | 5 | | user 2 | c2 | 7 | | user 2 | c3 | 2 | Ultimately, to get a matrix like the foll...

Fatal error in launcher: Unable to create process using “”C:Program Files (x86)Python33python.exe“ ”C:Program Files (x86)Python33pip.exe“” Searching the net this seems to be a problem caused by spaces in the Python installation path. How do I get pip to work without having to reinstall everything in a path without spaces ? pip Have you tried using short path names for program files(X86)? e.g. "C:PROGRA~2Python33python.exe" – Shadow9043 Jul 8 '14 at 8:55 The problem is that when pip is installed, it will use the original, long name it gets from the system. See my answer below. – Archimedix Jul 9 '14 at 9:57 pip Reproducable with Python 2.7 and Windo...

python google geolocation api using wifi mac I'm trying to use Google's API for geolocation giving wifi data to determine location. This is their intro. And this is my code @author: Keith """ import requests payload = { "considerIp": "false", "wifiAccessPoints": [ { "macAddress": "00:25:9c:cf:1c:ac", "signalStrength": -43, "signalToNoiseRatio": 0 }, { "macAddress": "00:25:9c:cf:1c:ad", "signalStrength": -55, "signalToNoiseRatio": 0 } ], 'key':'<MyAPIKey>' } r = requests.post('https://www.googleapis.com/geolocation/v1/geolocate', params=payload) print(r.text) This is the output { "location": { "lat": 32.3643098, "lng": -88.703656 }, "accuracy": 6061.0 } The request ignored all of the payload except t...

How to manage Numpy arrays in Pandas DataFrames Let's assume one has a DataFrame with some integers values and some arrays defined somehow: df = pd.DataFrame(np.random.randint(0,100,size=(5, 1)), columns=['rand_int']) array_a = np.arange(5) array_b = np.arange(7) df['array_a'] = df['rand_int'].apply(lambda x: array_a[:x]) df['array_b'] = df['rand_int'].apply(lambda x: array_b[:x]) Some questions which can help me understand how to manage Numpy arrays with Pandas DataFrames: array_diff So you want to multiply both array a and array b by the corresponding value in rand_int? – user3483203 Jul 2 at 7:20 Not only that, but also define another column in df which is the np.setdiff1d between the rows in array_a and array_b . Thank you – espogian ...

Creating folder in python [duplicate] This question already has an answer here: I want to create a folder after an hour of the current time in python. I know how to get the current time and date and to create a folder. But how to create a folder at a time specified by me. Any help would be appreciated. from datetime import datetime from datetime import timedelta import os while True: now = datetime.now () #print(now.strftime("%H:%M:%S")) y = datetime.now () + timedelta (hours = 1) #print(y.strftime("%H:%M:%S")) if now== y: os.makedirs (y.strftime ("%H/%M/%S")) will this work? EDIT :- I have to run the code continuously i.e. creating folders at every instant of time This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question. If you plan on doing something else in the mean while (I guess you do) you will need some thread to...