Sebastien Rousseau
Tuntube mu ›

Generative AI in 2023: How It Works, Where It Lands

Injiniyan transformer, benchmark ɗin samfuran 2023, amfani a ayyukan kudi, da tambayoyin gudanarwa da ba za a iya jinkirta su ba.

10 minti karatu
Banner for: Generative AI in 2023: How It Works, Where It Lands

Abstract neural network visualisation in blue and purple tones representing AI processing.class="img-fluid clearfix"

Taƙaitaccen Rahoto / Muhimman Darasi

  • Ginin da ya canza komai. Takarda transformer na 2017 ta gabatar da self-attention: hanya da ke lissafta nauyin dacewa tsakanin kowane nau'i biyu na tokens a cikin shigarwa, ta maye gurbin sarrafa jerin RNNs da ayyukan matrix da za a iya gudanar da su tare. Kowane babban ƙirar harshe a cikin 2023 bambance ne na transformer (Vaswani et al., 2017).
  • GPT-4 a matsayin benchmark na 2023. An saki Maris 2023, GPT-4 ya samu matsayi na percentile na 90 a jarrabawar bar na Amurka, na 99 a GRE Verbal, kuma ya nuna tunani mai matakai da yawa a cikin takardun dogon. Ya kafa ma'aunin iyawa da ƙirar da suka biyo baya suka nufi cimma ko wuce (OpenAI, 2023).
  • Ƙirar open-weight sun dimokuratiyya damar shiga. Llama 2 na Meta (Yuli 2023) da Mistral 7B na Mistral AI (Satumba 2023) sun nuna cewa ƙirar da suke gasa da iyawar GPT-3.5 na iya gudu akan kayan aikin sirri — suna magance buƙatun zama na bayanai na masana'antun da aka tsara.
  • Gwajin sabis na kuɗi a cikin 2023. Ɗaukar aikin mai faɗi a ƙarshen 2023 sun haɗa da nazarin kwantiragin shari'a (binciken DocLLM na JPMorgan), sa ido kan canjin doka, da kayan aikin ƙarfin masu haɓaka. Goldman Sachs ya ba da rahoton amfani da ciki na mataimakan coding AI a cikin masu haɓaka 10,000.
  • Hallucination shingen samarwa ne. LLMs suna samar da sakamakon da ya zama kamar mai inganci amma kuskure ne a ƙididdiga masu muhimmanci. A cikin yanayin amfani da aka tsara — yanke shawarar bashi, ra'ayoyin bin doka, bayyanar abokin ciniki — hallucination ba lahani ne na ado ba; haɗari ne na doka da nauyi wanda ke buƙatar matakan rage haɗari na gini kamar retrieval-augmented generation (RAG).

Yadda Ginin Transformer ke Aiki #

Kowane muhimmin ƙirar harshe da aka ɗauka a cikin 2023 — GPT-4, Claude 2, Llama 2, Mistral, Falcon — an gina shi akan ginin transformer da aka gabatar a cikin takarda ta 2017 "Attention Is All You Need." Fahimtar hanyar tsakiya tana bayyana dalilin da ya sa waɗannan ƙirar ke aiki da inda suke gazawa.

Tokens da embeddings. Ƙira tana fara da raba rubutun shigarwa zuwa tokens na ƙaramin kalma (yawanci ta amfani da byte-pair encoding). An taƙaita kowane token zuwa vector mai girma mai yawa (embedding) wanda ke ɓoye alaƙar ma'anarsa da sauran tokens, da aka koya a lokacin pre-training.

Self-attention. Don kowane token, ƙira tana lissafta vectors guda uku: Query (abin da token ɗin nan ke nema), Key (abin da token ɗin nan ke bayarwa), da Value (abin da token ɗin nan ke ba da gudummawa). Ana lissafta maki na attention ta hanyar ɗaukar samfurin ɗigo na kowane Query da duk Keys, amfani da softmax don samar da nauyi, da taƙaitaccen Values da aka nauyi da waɗancan maki. Wannan yana nufin kowane token yana kula da kowane token ɗaya a cikin taga mahallin a lokaci guda — hanyar da ke ba transformers ikon su na magance dogon zango na dogaro.

Multi-head attention. Heads ɗin attention da yawa suna gudana tare, kowane ɗaya yana koyan nau'ikan dangantaka daban-daban (na waka-waka, na ma'ana, na matsayi). An haɗa sakamakonsu kuma aka tsara su a layi.

Layers na feed-forward. Bayan attention, kowane matsayi yana wucewa ta canje-canje biyu na layi tare da aiki mai ƙarfi mara layi. Wannan layer yana aiwatar da lissafin kowane token a kansa, yana kama canje-canjen fasalin gida.

Girma. GPT-4 an kiyasta yana da sama da ƙididdigar tiriliyon guda ɗaya (ba a tabbatar ba ta OpenAI). Llama 2 70B yana amfani da biliyan 70. Mistral 7B yana amfani da biliyan 7, tare da grouped-query attention da sliding window attention don inganci. Manyan ƙirar yawanci suna nuna mafi kyau zero-shot da few-shot tunani — iyawar da ke tasowa wanda ke sa su amfani ga ayyukan da ba a horar da su a fili ba.

Shimfidar Ƙira ta 2023 #

2023 ya samar da manyan ayyukan ƙira da yawa fiye da kowace shekara da ta gabata:

GPT-4 (OpenAI, Maris 2023). Multimodal (shigarwa rubutu + hoto), taga mahallin har zuwa tokens 128,000 a cikin bambancin GPT-4 Turbo na gaba, ƙarfi mai kyau na tunani mai matakai da yawa. Ya kafa benchmark don ayyukan yankin ƙwararru.

Claude 2 (Anthropic, Yuli 2023). Taga mahallin token 100,000 (mafi tsawo a lokacin ƙaddamarwa), ƙarfi mai kyau akan ayyukan takarda mai tsawo kamar nazarin kwantiragi da nazarin doka. Horar da Constitutional AI don rage sakamakon cutarwa.

Llama 2 (Meta, Yuli 2023). Sakin open-weight a bambancin ƙididdigar 7B, 13B, 34B, da 70B. An ba da izinin amfani da kasuwancin. Ya ba da damar ɗaukar aikin kan-wurin don masana'antun da aka tsara. Ya haifar da bambance-bambancen fine-tuned da yawa (Code Llama, Vicuna, WizardLM).

Mistral 7B (Mistral AI, Satumba 2023). Ƙididdigar biliyan 7 wanda ya wuce Llama 2 13B akan mafi yawan benchmarks. Grouped-query attention da sliding window attention suna rage farashin inference. Farkon muhimmin ƙirar iyakar Turai, mai dacewa a lura da mahallin GDPR da EU AI Act.

Falcon 180B (TII, Satumba 2023). Ƙirar open-weight na ƙididdigar biliyan 180, an horar da shi akan tokens tiriliyan 3.5 na bayanai na RefinedWeb. Ya nuna cewa ƙirar open-weight na iya kusantar ma'aunin GPT-4.

Inda Generative AI ya Sauka Da Farko a Sabis na Kuɗi #

A ƙarshen 2023, cibiyoyin kuɗi sun koma daga gwajin ciki zuwa shirye-shiryen gwaji na tsari a cikin yanayin amfani daban-daban da yawa:

Ƙarfin aikin masu haɓaka. Kayan aikin ƙirƙirar code (GitHub Copilot, Amazon CodeWhisperer, ƙirar fine-tuned na ciki) sun zama mafi yawan nau'in da aka ɗauka. Goldman Sachs ya ba da rahoton cewa masu haɓaka 10,000 suna da damar samun taimakon coding na AI. Morgan Stanley ya ɗauki GPT-4 a ciki don taimaka wa masu ba da shawara na kuɗi wajen dawo da bayanai daga tushen ilimi na takardun 100,000.

Sarrafa takardun shari'a da doka. Tsautsaye na karatun kwantiragi, sa ido kan canjin doka, da taswirar bin doka sun kasance gwajin da suka fi kima. Binciken JPMorgan akan DocLLM ya nuna cewa ƙirar harshe masu sani da tsarin takarda sun fi LLMs gabaɗaya a cikin ayyukan fahimtar takardu na kuɗi.

Ƙarfawa sabis na abokin ciniki. Bankuna sun ɗauki mataimaka masu amfani da LLM don tambayoyin abokin ciniki na layi na farko, tare da ɗaukaka ɗan adam don shawara da aka tsara. Ƙa'idojin muhimmanci: ƙira ba ta iya ba da shawara da aka tsara ba, ba za ta hallucinate sharuɗɗan kayan aiki ba, kuma dole ne a iya dubawa.

Ƙirƙirar labari na KYC da AML. Taƙaita tsare-tsaren ma'amala mai rikitarwa da bayanin martaba na abokin ciniki don bitar mai nazari — maye gurbin abin da ya kasance aikin rubutun hannu — ya fito a matsayin yanayin amfani mai inganci tare da ƙarancin haɗarin hallucination domin ƙira tana taƙaita bayanan da aka samar maimakon ƙirƙirar iƙirari na sabon abu.

Haɗarori da Samarwa ta Bayyana #

Canza daga nunawa zuwa samarwa a cikin sabis na kuɗi ya nuna jerin haɗarori waɗanda suke buƙatar amsa na gini:

Hallucination. LLMs suna samar da sakamakon da ya zama kamar mai amincewa amma kuskure ne a ƙididdigar da ke bambanta da nau'in aiki da ƙira. Akan ayyukan tunawa da gaskiya, har da GPT-4 yana hallucinate a ƙididdigar da ba za a yarda ba don ra'ayoyin bin doka ko bayyanar bashi. Rage haɗari na farko shine retrieval-augmented generation (RAG): ƙasa sakamakon ƙira a cikin takardun da aka dawo da su, masu iya tabbatarwa maimakon dogaro da ilimin parametric kaɗai.

Prompt injection. Shigarwa masu adawa da aka saka a cikin takardun ko saƙonnin mai amfani na iya canza halayyar ƙira. A cikin sabis na kuɗi, inda LLMs ke sarrafa takardun da ba a amince da su ba (kwantiragogi, imel, shigarwar abokin ciniki), prompt injection haɗarin tsaron samarwa ne, ba na ka'idar ba.

Malalar bayanai. Ƙirar fine-tuned ko da aka ba da umurni akan bayanan sirri na iya sake yin waɗannan bayanan a cikin sakamakon — haɗarin abu don PII, matsayin kasuwancin, da bayanan abokin ciniki. Ana buƙatar sarrafawar gini (ɗaukar aikin sirri, sarrafa bayanai-a-mahallin, tace sakamakon), ba zaɓi ba.

Asalin ƙira da iya duba. Masu tsara doka suna tsammanin cibiyoyin kuɗi su bayyana yanke shawara na atomatik. LLM wanda ke samar da kimantawar bashi ba tare da sawun tunani da za a iya duba ba ya kasa buƙatun iya bayyana na GDPR Mataki na 22, tanadi na AI mai haɗari na EU AI Act, da jagorar haɗarin ƙira na FCA na yanzu.

Ilimi na tsufa. LLMs suna da cutoffs na horarwa. Ƙira da aka horar akan bayanai har zuwa farkon 2023 ba ta san canjin doka ba, yanke shawara akan ƙimar sha'ani, ko abubuwan da suka faru a kasuwannin bayan wannan kwanan wata — iyakancewa mai muhimmanci don yanayin amfani na bin doka na ainihi ko bayanin kasuwannin ba tare da RAG ko dawo da ainihi ba.

Buƙatun Mulki Kafin Ɗaukar Aiki #

Masu aiwatarwa na sabis na kuɗi da ke aiki a cikin 2023 ba su jira tabbacin doka kafin ɗaukar aiki ba — amma cibiyoyin da ke jagoranci sun ɗauki tsare-tsaren sarrafa haɗarin ƙira (MRM) da aka daidaita daga jagorar SR 11-7 da SS3/18:

Ƙidayar ƙira da takardu. LLMs da aka ɗauka don ayyukan kasuwanci suna buƙatar takardu na asalin bayanai na horarwa, hanyar fine-tuning, yanayin kuskure da aka sani, da aiki akan ƙungiyoyin tabbatarwa masu musamman ga yankin.

Duba wurare na ɗan adam a cikin madauri. Don sakamakon da aka tsara (yanke shawara akan bashi, ra'ayoyin bin doka, bayyanar abokin ciniki), bitar ɗan adam ta kasance tilas a cikin 2023. An yi amfani da atomatik don rubuta kuma taƙaita; sa hannu na ƙarshe ya kasance ɗan adam.

Haɗarin mai sayarwa. Amfani da API ƙira na ɓangare na uku (OpenAI, Anthropic, Google) yana gabatar da haɗarin tattarawa mai sayarwa, haɗarin zama bayanai, da haɗarin canjin ƙira (masu ba da sabis na iya sabunta ƙirar a hankali). Yarjejeniyoyin kasuwancin da ɗaukar aikin sirri suna magance waɗannan a bangare.

Hulɗa da doka. FCA, PRA, ECB, da FINRA duk sun fitar da takardun ko jawabai kan mulkin AI a cikin 2023. Sakon da ya yi daidai: tsare-tsaren haɗarin ƙira na yanzu suna amfani da AI, kuma kamfanoni ya kamata su kasance masu ɗaukar matakin farko a rubuta hanyarsu ta mulki kafin jagorar hukuma.

Tambayoyi Da Ake Yawan Yi #

Menene bambancin tsakanin babban ƙirar harshe da ƙirar tushe?

Babban ƙirar harshe (LLM) ƙira ce da aka horar akan bayanan rubutu a ma'auni don hasashen da ƙirƙirar harshe. Ƙirar tushe kalma ce mai faɗi don kowace babbar ƙira da aka yi pre-trained wanda za a iya daidaita ta (fine-tuned ko da aka ba ta umurni) don ayyukan downstream da yawa — ciki har da LLMs amma har da ƙirar hangen nesa, ƙirar code, da ƙirar multimodal. GPT-4 shine duka LLM da ƙirar tushe. DALL-E 3 ƙirar tushe ne amma ba LLM ba. A aikace, ana amfani da kalmomin a musanya sau da yawa yayin da ake nufin tsarin ƙirƙirar rubutu.

Menene retrieval-augmented generation kuma dalilin da ya sa ya da muhimmanci ga sabis na kuɗi?

RAG yana haɗa ƙirar harshe da tsarin dawo: maimakon dogaro kawai akan ilimin parametric na ƙira (abin da ta koya a lokacin horarwa), RAG tana dawo da takardun da suka dace a lokacin inference kuma tana ba da su a matsayin mahallin. Wannan yana rage hallucination sosai a kan ayyukan gaskiya domin ƙira tana haɗa rubutun da aka samar maimakon tunawa da gaskiyar da aka koya. Don sabis na kuɗi, RAG tana ba da damar yanayin amfani kamar sa ido kan canjin doka (koyaushe tana dawo da dokoki na yanzu) da nazarin kwantiragi (tana ƙasa da ƙira a cikin rubutun kwantiragi na ainihi) wanda da zai zama mai karkata zuwa hallucination da yawa da tsarin ƙirƙira na tsarkakakke.

Yadda cibiyoyin kuɗi ya kamata su sarrafa EU AI Act dangane da ɗaukar aiki na generative AI a cikin 2023?

EU AI Act na yana cikin tsarin doka a cikin 2023 (Majalisar Turai ta zartar a Maris 2024, ta fara aiki Agusta 2024). Duk da haka, cibiyoyin da ke da ayyuka a EU ko abokan ciniki na EU sun riga suna kimanta tsare-tsarensu. Tsarin AI mai haɗari mai yawa a kimantawar bashi, yanke shawarar aikin yi, da kayan aikin muhimmanci suna buƙatar kimantawar bin doka, hanyoyin sa ido na ɗan adam, da shiga na dubawa. Ƙirar General-purpose AI (GPAI) — wanda ya haɗa da ƙirar tushe kamar GPT-4 — suna da matakin buƙatun nasu game da gaskiya da haɗarin tsarin. Kamfanonin da suka fara aiki na takardu da mulki a cikin 2023 sun fi dacewa don lokutan aiwatarwa.

Menene bambancin aikace-aikace tsakanin fine-tuning da prompt engineering don ɗaukar aikin LLM na kasuwanci?

Fine-tuning tana canza nauyoyin ƙira ta hanyar ci gaba da horarwa akan bayanan musamman ga yankin — tana koyawa ƙira ilimi sabon da tsare-tsaren hali. Yana buƙatar bayanan horarwa masu lakabi, kasafin lissafi, da kula da ci gaba yayin da ake sabunta ƙirar tushe. Prompt engineering (ciki har da misalan few-shot da system prompts) tana tsarawa hali a lokacin inference ba tare da canza nauyoyi ba — mafi sauri don aiwatarwa da sabuntawa, amma iyakance da abin da ƙirar tushe ta riga ta sani. Don yawancin ɗaukar aiki na sabis na kuɗi na 2023, RAG tare da prompt engineering shine mafitar farawa da aka fi so; fine-tuning an kiyaye don lokuta inda ƙira ta buƙatar koyan kalmomin mallakar mallakar ko ɗaukar tsare-tsaren sakamakon tsauri.

Manazarta #

Bita ta ƙarshe .

Bita ta ƙarshe .