(11주차) 5월16일

• 강의영상
• imports
• Conv2D
  • errors
  • test1
  • tf.keras.layers.Conv2D(1,kernel_size=(2,2)) 요약
  • test2
  • test3
  • test4
  • test5
  • test6
• hw

강의영상

imports

import tensorflow as tf 
import tensorflow.experimental.numpy as tnp 

import numpy as np

tf.experimental.numpy.experimental_enable_numpy_behavior()

Conv2D

errors

conv2d=tf.keras.layers.Conv2D(1,(2,2))

XXX= tnp.arange(1*4*4*1,dtype=tf.float64).reshape(1,4,4,1)
XXX

<tf.Tensor: shape=(1, 4, 4, 1), dtype=float64, numpy=
array([[[[ 0.],
         [ 1.],
         [ 2.],
         [ 3.]],

        [[ 4.],
         [ 5.],
         [ 6.],
         [ 7.]],

        [[ 8.],
         [ 9.],
         [10.],
         [11.]],

        [[12.],
         [13.],
         [14.],
         [15.]]]])>

conv2d(XXX)

<tf.Tensor: shape=(1, 3, 3, 1), dtype=float32, numpy=
array([[[[ 1.9133414],
         [ 3.3166614],
         [ 4.719981 ]],

        [[ 7.526621 ],
         [ 8.92994  ],
         [10.333261 ]],

        [[13.139901 ],
         [14.543221 ],
         [15.946542 ]]]], dtype=float32)>

- 에러가 생기는경우

XXX= tnp.arange(1*4*4,dtype=tf.float64).reshape(1,4,4)
XXX

<tf.Tensor: shape=(1, 4, 4), dtype=float64, numpy=
array([[[ 0.,  1.,  2.,  3.],
        [ 4.,  5.,  6.,  7.],
        [ 8.,  9., 10., 11.],
        [12., 13., 14., 15.]]])>

conv2d(XXX)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Input In [14], in <cell line: 1>()
----> 1 conv2d(XXX)

File ~/anaconda3/envs/tfgpu/lib/python3.10/site-packages/keras/utils/traceback_utils.py:67, in filter_traceback.<locals>.error_handler(*args, **kwargs)
     65 except Exception as e:  # pylint: disable=broad-except
     66   filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67   raise e.with_traceback(filtered_tb) from None
     68 finally:
     69   del filtered_tb

File ~/anaconda3/envs/tfgpu/lib/python3.10/site-packages/keras/engine/input_spec.py:227, in assert_input_compatibility(input_spec, inputs, layer_name)
    225   ndim = x.shape.rank
    226   if ndim is not None and ndim < spec.min_ndim:
--> 227     raise ValueError(f'Input {input_index} of layer "{layer_name}" '
    228                      'is incompatible with the layer: '
    229                      f'expected min_ndim={spec.min_ndim}, '
    230                      f'found ndim={ndim}. '
    231                      f'Full shape received: {tuple(shape)}')
    232 # Check dtype.
    233 if spec.dtype is not None:

ValueError: Input 0 of layer "conv2d" is incompatible with the layer: expected min_ndim=4, found ndim=3. Full shape received: (1, 4, 4)

XXX= tnp.arange(1*4*4*1).reshape(1,4,4,1)
XXX

<tf.Tensor: shape=(1, 4, 4, 1), dtype=int64, numpy=
array([[[[ 0],
         [ 1],
         [ 2],
         [ 3]],

        [[ 4],
         [ 5],
         [ 6],
         [ 7]],

        [[ 8],
         [ 9],
         [10],
         [11]],

        [[12],
         [13],
         [14],
         [15]]]])>

conv2d(XXX)

---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
Input In [17], in <cell line: 1>()
----> 1 conv2d(XXX)

File ~/anaconda3/envs/tfgpu/lib/python3.10/site-packages/keras/utils/traceback_utils.py:67, in filter_traceback.<locals>.error_handler(*args, **kwargs)
     65 except Exception as e:  # pylint: disable=broad-except
     66   filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67   raise e.with_traceback(filtered_tb) from None
     68 finally:
     69   del filtered_tb

File ~/anaconda3/envs/tfgpu/lib/python3.10/site-packages/tensorflow/python/framework/ops.py:7107, in raise_from_not_ok_status(e, name)
   7105 def raise_from_not_ok_status(e, name):
   7106   e.message += (" name: " + name if name is not None else "")
-> 7107   raise core._status_to_exception(e) from None

InvalidArgumentError: Exception encountered when calling layer "conv2d" (type Conv2D).

Value for attr 'T' of int64 is not in the list of allowed values: half, bfloat16, float, double, int32
	; NodeDef: {{node Conv2D}}; Op<name=Conv2D; signature=input:T, filter:T -> output:T; attr=T:type,allowed=[DT_HALF, DT_BFLOAT16, DT_FLOAT, DT_DOUBLE, DT_INT32]; attr=strides:list(int); attr=use_cudnn_on_gpu:bool,default=true; attr=padding:string,allowed=["SAME", "VALID", "EXPLICIT"]; attr=explicit_paddings:list(int),default=[]; attr=data_format:string,default="NHWC",allowed=["NHWC", "NCHW"]; attr=dilations:list(int),default=[1, 1, 1, 1]> [Op:Conv2D]

Call arguments received:
  • inputs=tf.Tensor(shape=(1, 4, 4, 1), dtype=int64)

- float16,32,64는 에러가 나지 않음

XXX= tnp.arange(1*4*4*1,dtype=tf.float16).reshape(1,4,4,1)
conv2d(XXX)

<tf.Tensor: shape=(1, 3, 3, 1), dtype=float32, numpy=
array([[[[ 1.9133414],
         [ 3.3166614],
         [ 4.719981 ]],

        [[ 7.526621 ],
         [ 8.92994  ],
         [10.333261 ]],

        [[13.139901 ],
         [14.543221 ],
         [15.946542 ]]]], dtype=float32)>

XXX= tnp.arange(1*4*4*1,dtype=tf.float32).reshape(1,4,4,1)
conv2d(XXX)

<tf.Tensor: shape=(1, 3, 3, 1), dtype=float32, numpy=
array([[[[ 1.9133414],
         [ 3.3166614],
         [ 4.719981 ]],

        [[ 7.526621 ],
         [ 8.92994  ],
         [10.333261 ]],

        [[13.139901 ],
         [14.543221 ],
         [15.946542 ]]]], dtype=float32)>

XXX= tnp.arange(1*4*4*1,dtype=tf.float64).reshape(1,4,4,1)
conv2d(XXX)

<tf.Tensor: shape=(1, 3, 3, 1), dtype=float32, numpy=
array([[[[ 1.9133414],
         [ 3.3166614],
         [ 4.719981 ]],

        [[ 7.526621 ],
         [ 8.92994  ],
         [10.333261 ]],

        [[13.139901 ],
         [14.543221 ],
         [15.946542 ]]]], dtype=float32)>

test1

- 첫시도

XXX= tnp.arange(1*4*4*1,dtype=tf.float64).reshape(1,4,4,1)
XXX.reshape(1,4,4)

<tf.Tensor: shape=(1, 4, 4), dtype=float64, numpy=
array([[[ 0.,  1.,  2.,  3.],
        [ 4.,  5.,  6.,  7.],
        [ 8.,  9., 10., 11.],
        [12., 13., 14., 15.]]])>

conv2d(XXX).reshape(1,3,3)

<tf.Tensor: shape=(1, 3, 3), dtype=float32, numpy=
array([[[ 1.9133414,  3.3166614,  4.719981 ],
        [ 7.526621 ,  8.92994  , 10.333261 ],
        [13.139901 , 14.543221 , 15.946542 ]]], dtype=float32)>

• XXX에서 conv2d(XXX)로 가는 규칙이 무엇일까?

- 코드의정리 -> 랜덤으로 바뀌는 부분이 있어서 seed고정

tf.random.set_seed(43052)
XXX= tnp.arange(1*4*4*1,dtype=tf.float64).reshape(1,4,4,1)
conv2d = tf.keras.layers.Conv2D(1,(2,2))

print(XXX.reshape(1,4,4))
print(conv2d(XXX).reshape(1,3,3))

tf.Tensor(
[[[ 0.  1.  2.  3.]
  [ 4.  5.  6.  7.]
  [ 8.  9. 10. 11.]
  [12. 13. 14. 15.]]], shape=(1, 4, 4), dtype=float64)
tf.Tensor(
[[[ -4.125754   -5.312817   -6.4998803]
  [ -8.874006  -10.0610695 -11.248133 ]
  [-13.622259  -14.809322  -15.996386 ]]], shape=(1, 3, 3), dtype=float32)

tf.reshape(conv2d.weights[0],(2,2))

<tf.Tensor: shape=(2, 2), dtype=float32, numpy=
array([[-0.13014299, -0.23927206],
       [-0.20175874, -0.6158894 ]], dtype=float32)>

0 * -0.13014299 + 1 * -0.23927206 + 4 * -0.20175874 + 5 * -0.6158894

-4.1257540200000005

- 결과 관찰하는 것이 힘드니까 에이트를 조정해보자.

conv2d.get_weights()

[array([[[[-0.13014299]],
 
         [[-0.23927206]]],
 
 
        [[[-0.20175874]],
 
         [[-0.6158894 ]]]], dtype=float32),
 array([0.], dtype=float32)]

conv2d.get_weights()[0].shape

(2, 2, 1, 1)

w= np.array([1/4,1/4,1/4,1/4],dtype=np.float32).reshape(2,2,1,1)

conv2d.get_weights()[1].shape

(1,)

b= np.array([3],dtype=np.float32)

conv2d.set_weights([w,b])

conv2d.get_weights()

[array([[[[0.25]],
 
         [[0.25]]],
 
 
        [[[0.25]],
 
         [[0.25]]]], dtype=float32),
 array([3.], dtype=float32)]

print(XXX.reshape(1,4,4))
print(conv2d(XXX).reshape(1,3,3))

tf.Tensor(
[[[ 0.  1.  2.  3.]
  [ 4.  5.  6.  7.]
  [ 8.  9. 10. 11.]
  [12. 13. 14. 15.]]], shape=(1, 4, 4), dtype=float64)
tf.Tensor(
[[[ 5.5  6.5  7.5]
  [ 9.5 10.5 11.5]
  [13.5 14.5 15.5]]], shape=(1, 3, 3), dtype=float32)

(0+1+4+5)/4 +3, (1+2+5+6)/4 +3, (2+3+6+7)/4 +3 

(5.5, 6.5, 7.5)

tf.keras.layers.Conv2D(1,kernel_size=(2,2)) 요약

- 요약

(1) size=(2,2)인 윈도우를 만듬.

(2) XXX에 윈도우를 통과시켜서 (2,2)크기의 sub XXX 를 얻음. sub XXX의 각 원소에 conv2d.weights[0]의 각 원소를 element-wise하게 곱한다.

(3) (2)의 결과를 모두 더한다. 그리고 그 결과에 다시 conv2d.weights[1]을 수행

(4) 윈도우를 이동시키면서 반복!

test2

- XXX의 관측치가 여러개라면?

XXX = tnp.arange(2*4*4*1,dtype=tf.float64).reshape(2,4,4,1)
XXX.reshape(2,4,4)

<tf.Tensor: shape=(2, 4, 4), dtype=float64, numpy=
array([[[ 0.,  1.,  2.,  3.],
        [ 4.,  5.,  6.,  7.],
        [ 8.,  9., 10., 11.],
        [12., 13., 14., 15.]],

       [[16., 17., 18., 19.],
        [20., 21., 22., 23.],
        [24., 25., 26., 27.],
        [28., 29., 30., 31.]]])>

conv2d(XXX).reshape(2,3,3)

<tf.Tensor: shape=(2, 3, 3), dtype=float32, numpy=
array([[[ 5.5,  6.5,  7.5],
        [ 9.5, 10.5, 11.5],
        [13.5, 14.5, 15.5]],

       [[21.5, 22.5, 23.5],
        [25.5, 26.5, 27.5],
        [29.5, 30.5, 31.5]]], dtype=float32)>

(16+17+20+21)/4 + 3 

21.5

test3

- 윈도우의 크기가 (3,3)이라면?

XXX = tnp.array([1]*1*4*4*1,dtype=tf.float64).reshape(1,4,4,1)
XXX.reshape(1,4,4)

<tf.Tensor: shape=(1, 4, 4), dtype=float64, numpy=
array([[[1., 1., 1., 1.],
        [1., 1., 1., 1.],
        [1., 1., 1., 1.],
        [1., 1., 1., 1.]]])>

• 이렇게하면 conv의 결과과 weight의 sum이 나오므로 확인하기 쉽다.

conv2d = tf.keras.layers.Conv2D(1,(3,3))
conv2d(XXX).reshape(1,2,2)

<tf.Tensor: shape=(1, 2, 2), dtype=float32, numpy=
array([[[1.7760725, 1.7760725],
        [1.7760725, 1.7760725]]], dtype=float32)>

tf.reduce_sum(tf.reshape(conv2d.weights[0],(3,3)))

<tf.Tensor: shape=(), dtype=float32, numpy=1.7760723>

test4

XXX = tf.constant([[3,3,2,1,0],[0,0,1,3,1],[3,1,2,2,3],[2,0,0,2,2],[2,0,0,0,1]],dtype=tf.float64).reshape(1,5,5,1)
XXX

<tf.Tensor: shape=(1, 5, 5, 1), dtype=float64, numpy=
array([[[[3.],
         [3.],
         [2.],
         [1.],
         [0.]],

        [[0.],
         [0.],
         [1.],
         [3.],
         [1.]],

        [[3.],
         [1.],
         [2.],
         [2.],
         [3.]],

        [[2.],
         [0.],
         [0.],
         [2.],
         [2.]],

        [[2.],
         [0.],
         [0.],
         [0.],
         [1.]]]])>

conv2d = tf.keras.layers.Conv2D(1,(3,3))

conv2d.get_weights()

[]

conv2d(XXX)

<tf.Tensor: shape=(1, 3, 3, 1), dtype=float32, numpy=
array([[[[-0.75701153],
         [-1.5324322 ],
         [-2.275369  ]],

        [[-0.07609195],
         [-0.71050227],
         [ 0.36970985]],

        [[-0.39251935],
         [ 0.39295316],
         [-1.08425   ]]]], dtype=float32)>

conv2d.get_weights()[0].shape

(3, 3, 1, 1)

_w = np.array([[0,1,2],[2,2,0],[0,1,2]],dtype=np.float32).reshape(3, 3, 1, 1)
_b = np.array([0],dtype=np.float32)

conv2d.set_weights([_w,_b])

conv2d(XXX).reshape(1,3,3)

<tf.Tensor: shape=(1, 3, 3), dtype=float32, numpy=
array([[[12., 12., 17.],
        [10., 17., 19.],
        [ 9.,  6., 14.]]], dtype=float32)>

test5

- conv2의 채널이 여러개라면?

XXX = tnp.array([1]*1*2*2*1,dtype=tf.float64).reshape(1,2,2,1)
XXX.reshape(1,2,2)

<tf.Tensor: shape=(1, 2, 2), dtype=float64, numpy=
array([[[1., 1.],
        [1., 1.]]])>

conv2d = tf.keras.layers.Conv2D(4,(2,2))

conv2d(XXX)

<tf.Tensor: shape=(1, 1, 1, 4), dtype=float32, numpy=
array([[[[-0.44725284, -0.2634663 ,  0.17193237,  1.0976446 ]]]],
      dtype=float32)>

conv2d(XXX).shape

TensorShape([1, 1, 1, 4])

• conv2d : (2,2,1) -> (1,1,4) 로 바꾸는 변환

np.array(conv2d.weights[0])[...,0].sum() ## conv2d(XXX)의 첫번째 채널 

-0.4472528

np.array(conv2d.weights[0])[...,1].sum() ## conv2d(XXX)의 두번째 채널 

-0.2634663

np.array(conv2d.weights[0])[...,2].sum() ## conv2d(XXX)의 세번째 채널 

0.17193237

np.array(conv2d.weights[0])[...,3].sum() ## conv2d(XXX)의 네번째 채널 

1.0976446

test6

- X의 채널도 여러개이고, conv2의 채널도 여러개라면?

XXX = tnp.array([1]*1*2*2*3,dtype=tf.float64).reshape(1,2,2,3)

conv2d = tf.keras.layers.Conv2D(4,(2,2))

conv2d(XXX)

<tf.Tensor: shape=(1, 1, 1, 4), dtype=float32, numpy=
array([[[[-0.21142393, -0.2087717 ,  1.7372127 ,  0.44387752]]]],
      dtype=float32)>

• conv2d: (2,2,3) -> (1,1,4) 로 만들어 주는 변환

conv2d.weights ## 처음 (2,2)는 window size // 3은 XXX의 채널수 // 4는 conv(XXX)의 채널수

[<tf.Variable 'conv2d_12/kernel:0' shape=(2, 2, 3, 4) dtype=float32, numpy=
 array([[[[ 0.05917424, -0.28489804,  0.11780715, -0.04244581],
          [ 0.17721492, -0.22674735,  0.4233629 ,  0.38828826],
          [-0.19064838, -0.32950848,  0.44972152, -0.27025908]],
 
         [[-0.27793118,  0.04875344,  0.15190583, -0.10116643],
          [ 0.00985605,  0.39692885, -0.36121044,  0.28199434],
          [ 0.11366147,  0.07311636,  0.43144172, -0.14816672]]],
 
 
        [[[ 0.21962047,  0.08759755,  0.05900085,  0.20004088],
          [ 0.05466127, -0.11544552,  0.12602174,  0.0126003 ],
          [-0.39480078, -0.2934072 ,  0.11134905,  0.01866451]],
 
         [[ 0.38467884,  0.02177629, -0.21719636, -0.42347163],
          [-0.38105386,  0.3438273 ,  0.03691953,  0.4573207 ],
          [ 0.01414302,  0.06923515,  0.40808922,  0.0704782 ]]]],
       dtype=float32)>,
 <tf.Variable 'conv2d_12/bias:0' shape=(4,) dtype=float32, numpy=array([0., 0., 0., 0.], dtype=float32)>]

print(conv2d(XXX)) 
print(np.array(conv2d.weights[0])[...,0].sum())
print(np.array(conv2d.weights[0])[...,1].sum())
print(np.array(conv2d.weights[0])[...,2].sum())
print(np.array(conv2d.weights[0])[...,3].sum())

tf.Tensor([[[[-0.21142393 -0.2087717   1.7372127   0.44387752]]]], shape=(1, 1, 1, 4), dtype=float32)
-0.21142393
-0.20877159
1.7372127
0.44387752

W_red = np.array(conv2d.weights[0])[...,0][...,0]
W_green = np.array(conv2d.weights[0])[...,0][...,1]
W_blue = np.array(conv2d.weights[0])[...,0][...,2]

(W_red+W_green+W_blue).sum() ### convXXX의 첫채널 결과 

-0.21142393

W_red = np.array(conv2d.weights[0])[...,1][...,0]
W_green = np.array(conv2d.weights[0])[...,1][...,1]
W_blue = np.array(conv2d.weights[0])[...,1][...,2]
(W_red+W_green+W_blue).sum() ### convXXX의 두번째 채널 결과 

-0.20877162

hw

아래와 같은 흑백이미지가 있다고 하자.

0 0 0 1 1 1 
0 0 0 1 1 1 
0 0 0 1 1 1 
0 0 0 1 1 1 
0 0 0 1 1 1
0 0 0 1 1 1

위의 이미지에 아래와 같은 weight를 가진 필터를 적용하여 convolution한 결과를 계산하라. (bias는 0으로 가정한다)

-1 1 
-1 1