728x90

๋จธ์‹ ๋Ÿฌ๋‹ 10

[RNN] 2. Vanialla RNN, LSTM(Long Short-Term Memory), GRU(Gated Recurrent Unit)

์ˆœ์ฐจ ๋ฐ์ดํ„ฐ๋ฅผ ์–ด๋–ป๊ฒŒ ์ˆœํ™˜ ์‹ ๊ฒฝ๋ง์„ ํ†ตํ•ด ์ฒ˜๋ฆฌํ•˜๋Š”์ง€ ์•Œ์•„๋ณด์ž. ๊ธฐ์–ต ์‹œ์Šคํ…œ ๋งŒ์•ฝ ์‹œ๋ฆฌ ๊ฐ™์€ ๊ฐœ์ธ ๋น„์„œ์˜ ๊ฒฝ์šฐ ์˜ฌ๋ฐ”๋ฅธ ๋Œ€๋‹ต์„ ํ•˜๋ ค๋ฉด ์ž…๋ ฅ์„ ๋ฐ›์„ ๋•Œ๋งˆ๋‹ค ๊ทธ ๋‚ด์šฉ์„ ๊ธฐ์–ตํ•ด์•ผ ํ•œ๋‹ค. ๋ฐ˜๋Œ€๋กœ ์ด์ „ ์ž…๋ ฅ์„ ๊ธฐ์–ตํ•˜์ง€ ์•Š๋Š” ์‹œ์Šคํ…œ์€ ๋ฌด๊ธฐ์–ต ์‹œ์Šคํ…œ์ด๋ผ ํ•œ๋‹ค. ์–•์€ ์‹ ๊ฒฝ๋ง(Shallow Neural Network)์ด ๋ฌด๊ธฐ์–ต ์‹œ์Šคํ…œ์˜ ์˜ˆ์‹œ n๋ฒˆ์งธ Time-Step์— ๋Œ€ํ•œ ๊ฒฐ๊ณผ๊ฐ€ ์ด์ „ ์ž…๋ ฅ์— ์˜ํ–ฅ ๋ฐ›์ง€ ์•Š๋Š”๋‹ค. ๊ธฐ๋ณธ์ ์ธ ์ˆœํ™˜ ์‹ ๊ฒฝ๋ง (Vanilla Recurrent Network) Vanilla RNN์˜ ๊ตฌ์กฐ๋Š” shallow NN ๊ตฌ์กฐ์— '์ˆœํ™˜(recurrent)'์ด ์ถ”๊ฐ€๋œ ๊ฒƒ์ด๋‹ค. ์ˆœํ™˜: n-1๋ฒˆ์งธ time step์ด n๋ฒˆ์งธ time step์œผ๋กœ ๋‹ค์‹œ ๋Œ์•„์˜ค๋Š” ๊ฒƒ ๊ธฐ์–ต ์‹œ์Šคํ…œ์ด๋ฏ€๋กœ RNN์˜ ์ถœ๋ ฅ์€ ์ด์ „์˜ ๋ชจ๋“  ์ž…๋ ฅ์— ์˜ํ–ฅ..

[RNN] ์ˆœํ™˜ ์‹ ๊ฒฝ๋ง (RNN, Recurrent Neural Network) - 1. ์ˆœ์ฐจ ๋ฐ์ดํ„ฐ

์ˆœํ™˜ ์‹ ๊ฒฝ๋ง์€ ์ž…๋ ฅ์ด๋‚˜ ์ถœ๋ ฅ์„ ์ˆœ์ฐจ ๋ฐ์ดํ„ฐ๋กœ ๋ฝ‘๊ธฐ ์œ„ํ•œ ์‹ ๊ฒฝ๋ง์ด๋‹ค. ๋”ฐ๋ผ์„œ ์ˆœ์ฐจ ๋ฐ์ดํ„ฐ(Sequential Data)์— ๋Œ€ํ•œ ์ดํ•ด๊ฐ€ ํ•„์š”ํ•˜๋‹ค. ์ˆœ์ฐจ๋ฐ์ดํ„ฐ๋ž€? ์ˆœ์„œ๊ฐ€ ๋‹ฌ๋ผ์งˆ ๊ฒฝ์šฐ ์˜๋ฏธ๊ฐ€ ์†์ƒ๋˜๋Š” ๋ฐ์ดํ„ฐ, ์ฆ‰ ์ˆœ์„œ๊ฐ€ ์˜๋ฏธ์žˆ๋Š” ๋ฐ์ดํ„ฐ ์‹œ๊ฐ„์  ์˜๋ฏธ๊ฐ€ ์žˆ๋Š” ๊ฒฝ์šฐ temporal sequence ์ผ์ •ํ•œ ์‹œ๊ฐ„์ฐจ๋ผ๋ฉด time series ex) DNA ์—ผ๊ธฐ ์„œ์—ด(sequential data), ๊ธฐ์˜จ ๋ณ€ํ™”(temporal sequence), ์ƒ˜ํ”Œ๋ง๋œ ์†Œ๋ฆฌ ์‹ ํ˜ธ(time series) โ€ป Resampling (๋ณด๊ฐ„ + ์ƒ˜ํ”Œ๋ง) Temporal Sequence ์‹ ํ˜ธ๋ฅผ Time Series๋กœ ๋ณ€ํ™˜ํ•˜๊ธฐ ์œ„ํ•ด ์‹ ํ˜ธ๋ฅผ ๋ณด๊ฐ„(interpolation)ํ•˜๊ณ  ์ด๋ฅผ ๊ท ์ผ ์‹œ๊ฐ„ ๊ฐ„๊ฒฉ์œผ๋กœ sampling ํ•œ๋‹ค. ์ˆœ์ฐจ ๋ฐ์ดํ„ฐ์™€ ์‹ฌ์ธต ์‹ ๊ฒฝ๋ง ์ˆœ..

[Deep Learning] Semantic Segmentation - Deconvolution, Upsampling

CNN(Convolutional Neural Network)์˜ convolution layer๋Š” convolution์„ ํ†ตํ•ด์„œ feature map์˜ ํฌ๊ธฐ๋ฅผ ์ค„์ธ๋‹ค. Deconvolution์€ CNN์˜ ์—ญ์—ฐ์‚ฐ์œผ๋กœ CNN๊ณผ ๋ฐ˜๋Œ€๋กœ feature map์˜ ํฌ๊ธฐ๋ฅผ ์ฆ๊ฐ€์‹œํ‚จ๋‹ค. ์ด๋Ÿฐ Deconvolution์€ ์–ด๋””์— ์“ฐ์ผ๊นŒ? ๋ฐ”๋กœ Semantic Segmantation์ด๋‹ค. Semantic Segmentation์ด๋ž€ Computer Vision Tasks๋“ค ์ค‘ ํ•˜๋‚˜์ด๋‹ค. ์œ„์˜ ๊ทธ๋ฆผ์— ์ž˜ ์„ค๋ช…๋˜์–ด ์žˆ๋Š”๋ฐ, Object Detection์€ ๋ฌผ์ฒด๊ฐ€ ์žˆ๋Š” ์œ„์น˜๋ฅผ ์ฐพ์•„ Bounding Box๋ฅผ ๊ทธ๋ฆฌ๋Š” ์ž‘์—…์ด๊ณ  Semantic Segmentation์ด๋ž€, ์ด๋ฏธ์ง€๋ฅผ ํ”ฝ์…€ ๋‹จ์œ„๋กœ ๊ตฌ๋ถ„ํ•ด ๊ฐ ํ”ฝ์…€์ด ์–ด๋–ค object class..

[๋…ผ๋ฌธ ๋ฆฌ๋ทฐ] Very Deep Convolutional Networks for Large-Scale Image Recognition ๋ฆฌ๋ทฐ, VGG Net

๋ฌด๋ ค 1๋…„์ „์— ์ •๋ฆฌํ•ด๋†“์€ ๋…ผ๋ฌธ ์˜ฌ๋ฆฌ๊ธฐ ใ…Žใ……ใ…Ž Image Recognition์— ์ž…๋ฌธํ•  ๋•Œ ์ข‹์€ ๋…ผ๋ฌธ์ด๋ผ๊ณ  ์ƒ๊ฐํ•œ๋‹ค. Very Deep Convolutional Networks for Large-Scale Image Recognition arxiv.org/abs/1409.1556 Very Deep Convolutional Networks for Large-Scale Image Recognition In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough eva..

[Deep Learning] Google Cloud TPU(CPU, GPU, NPU, TPU ๊ฐœ๋…)

CPU, GPU, NPU, TPU ๋ชจ๋‘ ๋ฐ์ดํ„ฐ๋ฅผ ์ฝ์–ด ์—ฐ์‚ฐ์ฒ˜๋ฆฌ ๊ธฐ๋Šฅ์„ ์ˆ˜ํ–‰ํ•˜๋Š” Processing Unit์ด๋‹ค. ํ•˜์ง€๋งŒ ๋”ฅ๋Ÿฌ๋‹ ๋ชจ๋ธ ํ•™์Šต ๋ฐ ์ถ”๋ก  ์‹œ CPU๋ณด๋‹ค๋Š” ๋‹ค๋ฅธ ์œ ๋‹›์„ ํ†ตํ•ด ์—ฐ์‚ฐ์„ ์ˆ˜ํ–‰ํ•˜๋Š” ์ผ์ด ๋งŽ๊ธฐ ๋•Œ๋ฌธ์— ๊ฐœ๋…์„ ์•Œ๊ณ ๊ฐ€์•ผํ•œ๋‹ค. CPU(Centralized Processing Unit) ์ฝ”์–ด๊ฐ€ ๋ช‡ ๊ฐœ์ธ์ง€, ํด๋ก ์Šคํ”ผ๋“œ๊ฐ€ ์–ผ๋งˆ๋‚˜ ๋˜๋Š”์ง€๊ฐ€ ์„ฑ๋Šฅ์˜ ์ง€ํ‘œ. ๊ณ ์„ฑ๋Šฅ ์ปดํ“จํ„ฐ์ผ์ˆ˜๋ก ์ฝ”์–ด์˜ ์ˆ˜๊ฐ€ ๋งŽ๋‹ค → multi-core, hyper-threading GPU(Graphic Processing Unit) CPU์™€ ๋น„์Šทํ•œ ์ ์ด ๋งŽ์ง€๋งŒ, CPU์™€ ๋‹ฌ๋ฆฌ ๊ทธ๋ž˜ํ”ฝ ๊ด€๋ จ ์ž‘์—…์ด๋‚˜ ๋ณ‘๋ ฌ์ฒ˜๋ฆฌ์— ๋งค์šฐ ํšจ๊ณผ์ ์ด๋‹ค. → ๋Œ€๊ทœ๋ชจ ๋ณ‘๋ ฌ ๊ณฑ์…ˆ NPU(Neural Processing Unit) ์ธ๊ณต์ง€๋Šฅ ๋ชจ๋ธ์˜ ์ธ๊ณต์‹ ๊ฒฝ๋ง ์—ฐ์‚ฐ์„ ํšจ์œจ์ ์œผ๋กœ..

[Deep Learning] CNN์˜ ๊ฐœ๋…, Object Detection

๋”ฅ๋Ÿฌ๋‹ ๋ชจ๋ธ ์ค‘ ํ•˜๋‚˜์ธ CNN(Convolutional Neural Network)์— ๋Œ€ํ•ด ์•Œ์•„๋ณด์ž. CNN์€ computer vision problem์—์„œ ๋งŽ์ด ์“ฐ์ธ๋‹ค. ํŠนํžˆ ๊ทธ ์ค‘ ๋งŽ์ด ํ™œ์šฉ๋˜๋Š” ๊ฒƒ์€ object detection์ด๋‹ค. Object Detection์ด๋ž€? Feature extraction(ํŠน์ง• ์ถ”์ถœ) ์ด๋ฏธ์ง€์—์„œ ๋Œ์–ด๋‚ผ ์ˆ˜ ์žˆ๋Š” ์œ ์šฉํ•œ feature ์ถ”์ถœ Bounding Box ์ƒ์„ฑ object๋ฅผ ๊ฐ์‹ธ๋Š” bounding box ์ƒ์„ฑ Class classification bounding box ์•ˆ์˜ object๊ฐ€ ์–ด๋–ค class์ธ์ง€ ๋ถ„๋ฅ˜ํ•˜๋Š” ๊ณผ์ • CNN(Convolutional Neural Network) image์˜ ํ˜•ํƒœ๋ฅผ ๋ณด์กดํ•˜๋„๋ก ํ–‰๋ ฌ ํ˜•ํƒœ์˜ ๋ฐ์ดํ„ฐ๋ฅผ ์ž…๋ ฅ๋ฐ›์•„ ์ •๋ณด ์†์‹ค์„ ๋ฐฉ์ง€ํ•˜๊ณ ,..

[Machine Learning] ๋จธ์‹ ๋Ÿฌ๋‹ workflow, ๋ฐ์ดํ„ฐ์…‹ ๋ถ„๋ฆฌ

ahnty0122.tistory.com/58 [Deep Learning] ๋”ฅ๋Ÿฌ๋‹/๋จธ์‹ ๋Ÿฌ๋‹/์ธ๊ณต์ง€๋Šฅ์˜ ์ฐจ์ด, ์ธ๊ณต์‹ ๊ฒฝ๋ง ๊ฐœ๋… ๋”ฅ๋Ÿฌ๋‹์— ๋Œ€ํ•ด ์ดํ•ดํ•˜๋ ค๋ฉด ๋จผ์ € ์ธ๊ณต์ง€๋Šฅ, ๋จธ์‹ ๋Ÿฌ๋‹, ๋”ฅ๋Ÿฌ๋‹์˜ ์ฐจ์ด์ ์„ ์•Œ์•„์•ผํ•œ๋‹ค. ์ธ๊ณต์ง€๋Šฅ(AI): ์‚ฌ๋žŒ์˜ ์ง€๋Šฅ์„ ๋ชจ๋ฐฉํ•ด ์‚ฌ๋žŒ์ด ํ•˜๋Š” ๋ณต์žกํ•œ ์ผ์„ ํ•  ์ˆ˜ ์žˆ๋Š” ๊ธฐ๊ณ„๋ฅผ ๋งŒ๋“œ๋Š” ๊ฒƒ ๋จธ์‹ ๋Ÿฌ๋‹(ML): ์ž… ahnty0122.tistory.com Machine Learning Workflow ์œ„์˜ ๊ธ€์—์„œ ์ธ๊ณต์‹ ๊ฒฝ๋ง์„ ํ•™์Šตํ•˜๋Š” ๊ณผ์ •์— ๋Œ€ํ•ด ์ ์–ด๋†จ๋Š”๋ฐ, ์ด ์ธ๊ณต์‹ ๊ฒฝ๋ง์„ ํ•™์Šตํ•˜๋Š” ๋ฐฉ๋ฒ•์— ๋Œ€ํ•ด ์•Œ์•„๋ณด์ž. ๋ฏธ๋ฆฌ ์ •ํ•ด์ง„, ๋‹ต์ด ์žˆ๋Š” ํŠธ๋ ˆ์ด๋‹ ๋ฐ์ดํ„ฐ๋กœ ๋จธ์‹ ๋Ÿฌ๋‹ ์•Œ๊ณ ๋ฆฌ์ฆ˜์„ ํ•™์Šตํ•ด ๋ชจ๋ธ์„ ์–ป๋Š” training(ํ•™์Šต)๊ณผ์ •์„ ๊ฑฐ์ณ ์˜ฌ๋ฐ”๋ฅด๊ฒŒ ํ•™์Šต์ด ๋˜๋Š”์ง€ ์ž„์˜์˜ ์ž…๋ ฅ๋ฐ์ดํ„ฐ๋ฅผ ํ†ตํ•ด ๊ฒฐ๊ณผ๋ฅผ ํ™•์ธํ•˜๋Š” test ๊ณผ์ •์„ ..

[Deep Learning] ๋”ฅ๋Ÿฌ๋‹/๋จธ์‹ ๋Ÿฌ๋‹/์ธ๊ณต์ง€๋Šฅ์˜ ์ฐจ์ด, ์ธ๊ณต์‹ ๊ฒฝ๋ง ๊ฐœ๋…

๋”ฅ๋Ÿฌ๋‹์— ๋Œ€ํ•ด ์ดํ•ดํ•˜๋ ค๋ฉด ๋จผ์ € ์ธ๊ณต์ง€๋Šฅ, ๋จธ์‹ ๋Ÿฌ๋‹, ๋”ฅ๋Ÿฌ๋‹์˜ ์ฐจ์ด์ ์„ ์•Œ์•„์•ผํ•œ๋‹ค. ์ธ๊ณต์ง€๋Šฅ(AI): ์‚ฌ๋žŒ์˜ ์ง€๋Šฅ์„ ๋ชจ๋ฐฉํ•ด ์‚ฌ๋žŒ์ด ํ•˜๋Š” ๋ณต์žกํ•œ ์ผ์„ ํ•  ์ˆ˜ ์žˆ๋Š” ๊ธฐ๊ณ„๋ฅผ ๋งŒ๋“œ๋Š” ๊ฒƒ ๋จธ์‹ ๋Ÿฌ๋‹(ML): ์ž…๋ ฅ๋˜์–ด ์žˆ๋Š” ๋ฐ์ดํ„ฐ๋ฅผ ํ†ตํ•ด ๊ฐ๊ฐ์˜ ์ƒํ™ฉ์— ๋งž๋„๋ก ํ•™์Šต์‹œํ‚ค๋Š” ๊ฒƒ ๋”ฅ๋Ÿฌ๋‹(DL): ๋จธ์‹ ๋Ÿฌ๋‹์œผ๋กœ ํ•™์Šตํ•œ ๋ถ€๋ถ„๋“ค์„ ๊ธฐ๋ฐ˜์œผ๋กœ ์Šค์Šค๋กœ ํŒ๋‹จํ•ด ์ƒˆ๋กญ๊ฒŒ ๋Œ€์ฒ˜ํ•˜๋Š” ๊ฒƒ (=end-to-end machine learning) ๋”ฅ๋Ÿฌ๋‹์ด end-to-end machine learning์ธ ์ด์œ ๋Š” ์ฒ˜์Œ๋ถ€ํ„ฐ ๋๊นŒ์ง€ ์ปดํ“จํ„ฐ๊ฐ€ ํ•™์Šตํ•˜๋Š” ๊ฒƒ์œผ๋กœ, ๋จธ์‹ ๋Ÿฌ๋‹์ฒ˜๋Ÿผ ์‚ฌ๋žŒ์ด ๋ฐ์ดํ„ฐ์—์„œ ์ผ๋ถ€ ํŠน์ง•์„ ๋ฝ‘์•„์„œ ๊ธฐ๊ณ„์—๊ฒŒ ์ „๋‹ฌํ•ด ํ•™์Šต์‹œํ‚ค๋Š” ๊ฒƒ์ด ์•„๋‹ˆ๋ผ ๋ฐ์ดํ„ฐ ์ž์ฒด๋ฅผ ์ปดํ“จํ„ฐ์— ์ „๋‹ฌํ•˜๋Š” ๋“ฑ ์‚ฌ๋žŒ์˜ ๊ฐœ์ž…์ด ์•„์˜ˆ ์—†๊ธฐ ๋•Œ๋ฌธ์ด๋‹ค. ์ธ๊ณต์ง€๋Šฅ, ๋จธ์‹ ๋Ÿฌ๋‹, ๋”ฅ๋Ÿฌ๋‹์˜ ์ •์˜๋ฅผ ..

728x90