Numpy […,None]

Multi tool use
Numpy […,None]
I have found myself needing to add features to existing numpy arrays which has led to a question around what the last portion of the following code is actually doing:
np.ones(shape=feature_set.shape)[...,None]
Set-up
As an example, let's say I wish to solve for linear regression parameter estimates by using numpy and solving:
Assume I have a feature set shape (50,1), a target variable of shape (50,), and I wish to use the shape of my target variable to add a column for intercept values.
It would look something like this:
# Create random target & feature set
y_train = np.random.randint(0,100, size = (50,))
feature_set = np.random.randint(0,100,size=(50,1))
# Build a set of 1s after shape of target variable
int_train = np.ones(shape=y_train.shape)[...,None]
# Able to then add int_train to feature set
X = np.concatenate((int_train, feature_set),1)
What I Think I Know
I see the difference in output when I include [...,None] vs when I leave it off. Here it is:
The second version returns an error around input arrays needing the same number of dimensions, and eventually I stumbled on the solution to use [...,None].
Main Question
While I see the output of [...,None] gives me what I want, I am struggling to find any information on what it is actually supposed to do. Can anybody walk me through what this code actually means, what the None argument is doing, etc?
Thank you!
np.ones((y_train.shape[0], 1))
np.ones((50,1))
None
2 Answers
2
The slice of [..., None]
consists of two "shortcuts":
[..., None]
The ellipsis literal component:
The dots (...) represent as many colons as needed to produce a complete indexing tuple. For example, if x is a rank 5 array (i.e., it has 5 axes), then
x[1,2,...]
x[1,2,:,:,:]
x[...,3]
x[:,:,:,:,3]
x[4,...,5,:]
x[4,:,:,5,:]
(Source)
The None
component:
None
numpy.newaxis
numpy.newaxis
The newaxis
object can be used in all slicing operations to create an axis of length one. newaxis
is an alias for ‘None’, and ‘None’ can be used in place of this with the same result.
newaxis
newaxis
(Source)
So, arr[..., None]
takes an array of dimension N
and "adds" a dimension "at the end" for a resulting array of dimension N+1
.
arr[..., None]
N
N+1
Example:
import numpy as np
x = np.array([[1,2,3],[4,5,6]])
print(x.shape) # (2, 3)
y = x[...,None]
print(y.shape) # (2, 3, 1)
z = x[:,:,np.newaxis]
print(z.shape) # (2, 3, 1)
a = np.expand_dims(x, axis=-1)
print(a.shape) # (2, 3, 1)
print((y == z).all()) # True
print((y == a).all()) # True
Consider this code:
np.ones(shape=(2,3))[...,None].shape
As you see the 'None' phrase change the (2,3) matrix to a (2,3,1) tensor. As a matter of fact it put the matrix in the LAST index of the tensor.
If you use
np.ones(shape=(2,3))[None, ...].shape
it put the matrix in the FIRST index of the tensor
By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.
You could also use
np.ones((y_train.shape[0], 1))
ornp.ones((50,1))
. TheNone
just adds a size one dimension.– hpaulj
Jul 2 at 0:46