Numpy […,None]

Multi tool use
Multi tool use


Numpy […,None]



I have found myself needing to add features to existing numpy arrays which has led to a question around what the last portion of the following code is actually doing:


np.ones(shape=feature_set.shape)[...,None]



Set-up



As an example, let's say I wish to solve for linear regression parameter estimates by using numpy and solving:



From Elements of Statistical Learning



Assume I have a feature set shape (50,1), a target variable of shape (50,), and I wish to use the shape of my target variable to add a column for intercept values.



It would look something like this:


# Create random target & feature set
y_train = np.random.randint(0,100, size = (50,))
feature_set = np.random.randint(0,100,size=(50,1))

# Build a set of 1s after shape of target variable
int_train = np.ones(shape=y_train.shape)[...,None]

# Able to then add int_train to feature set
X = np.concatenate((int_train, feature_set),1)



What I Think I Know



I see the difference in output when I include [...,None] vs when I leave it off. Here it is:



enter image description here



The second version returns an error around input arrays needing the same number of dimensions, and eventually I stumbled on the solution to use [...,None].



Main Question



While I see the output of [...,None] gives me what I want, I am struggling to find any information on what it is actually supposed to do. Can anybody walk me through what this code actually means, what the None argument is doing, etc?



Thank you!





You could also use np.ones((y_train.shape[0], 1)) or np.ones((50,1)). The None just adds a size one dimension.
– hpaulj
Jul 2 at 0:46



np.ones((y_train.shape[0], 1))


np.ones((50,1))


None




2 Answers
2



The slice of [..., None] consists of two "shortcuts":


[..., None]



The ellipsis literal component:



The dots (...) represent as many colons as needed to produce a complete indexing tuple. For example, if x is a rank 5 array (i.e., it has 5 axes), then


x[1,2,...]


x[1,2,:,:,:]


x[...,3]


x[:,:,:,:,3]


x[4,...,5,:]


x[4,:,:,5,:]



(Source)



The None component:


None



numpy.newaxis


numpy.newaxis



The newaxis object can be used in all slicing operations to create an axis of length one. newaxis is an alias for ‘None’, and ‘None’ can be used in place of this with the same result.


newaxis


newaxis



(Source)



So, arr[..., None] takes an array of dimension N and "adds" a dimension "at the end" for a resulting array of dimension N+1.


arr[..., None]


N


N+1



Example:


import numpy as np

x = np.array([[1,2,3],[4,5,6]])
print(x.shape) # (2, 3)

y = x[...,None]
print(y.shape) # (2, 3, 1)

z = x[:,:,np.newaxis]
print(z.shape) # (2, 3, 1)

a = np.expand_dims(x, axis=-1)
print(a.shape) # (2, 3, 1)

print((y == z).all()) # True
print((y == a).all()) # True



Consider this code:


np.ones(shape=(2,3))[...,None].shape



As you see the 'None' phrase change the (2,3) matrix to a (2,3,1) tensor. As a matter of fact it put the matrix in the LAST index of the tensor.



If you use


np.ones(shape=(2,3))[None, ...].shape



it put the matrix in the FIRST‌ index of the tensor






By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

3quTE4Q1SBMN2QW1V VAa6QUKQo82NGh,LNpTgRwO,umX8HVh
lTvKp8eRa sLgl5u4bUnc

Popular posts from this blog

Boo (programming language)

Rothschild family